AI Foundation Model Engineer

Job Title: AI Foundation Model Engineer

Location: Jersey City, NJ

About the Role

Seeking an experienced AI Foundation Model Engineer to design, build, deploy, and optimize enterprise-grade AI solutions powered by Large Language Models (LLMs), Generative AI, Retrieval-Augmented Generation (RAG), and agentic AI workflows. This role is responsible for developing scalable, secure, and production-ready AI applications while ensuring operational excellence, observability, governance, and compliance within enterprise environments.

The ideal candidate combines strong AI/ML engineering expertise with cloud-native software development and production deployment experience.

Key Responsibilities

• Design and develop LLM-powered applications including knowledge assistants, document intelligence platforms, workflow agents, summarization tools, and decision-support systems.
• Build Retrieval-Augmented Generation (RAG) pipelines using embeddings, semantic search, vector databases, chunking strategies, reranking, response grounding, and citation mechanisms.
• Fine-tune and optimize foundation models using techniques such as LoRA, PEFT, instruction tuning, transfer learning, knowledge distillation, quantization, and domain adaptation.
• Develop scalable APIs, microservices, model-serving infrastructure, and integration services across cloud, hybrid, and containerized environments.
• Optimize inference workloads for latency, throughput, token efficiency, scalability, reliability, cost optimization, and user experience.
• Implement observability solutions for AI applications including prompt logging, retrieval quality metrics, hallucination detection, model drift monitoring, service health, user feedback, and cost telemetry.
• Embed security, privacy, Responsible AI, model governance, and enterprise risk controls throughout the AI application lifecycle.
• Create production documentation, deployment guides, runbooks, release documentation, testing evidence, and audit-ready implementation artifacts.
• Collaborate with AI Researchers, Platform Engineers, Security, Product, Architecture, and Business teams to deliver enterprise AI capabilities.

Required Qualifications

• 7+ years of experience in AI/ML Engineering, Applied Machine Learning, Platform Engineering, Software Engineering, or related disciplines.
• Hands-on experience developing applications using Large Language Models (LLMs), Transformers, embeddings, Retrieval-Augmented Generation (RAG), semantic search, and Generative AI architectures.
• Strong Python development experience with frameworks such as PyTorch, TensorFlow, Hugging Face, LangChain, LlamaIndex, Semantic Kernel, or equivalent AI frameworks.
• Experience deploying production AI services using REST APIs, microservices, containers, Kubernetes, CI/CD pipelines, cloud-native services, and monitoring platforms.
• Strong understanding of model evaluation, fine-tuning, inference optimization, secure data handling, and AI application performance tuning.
• Experience working with cloud platforms and distributed AI workloads.
• Excellent problem-solving, software engineering, and collaboration skills.

Preferred Qualifications

• Experience within Banking, Financial Services, FinTech, Risk Management, Compliance, Financial Crime, Operations, or Enterprise Technology.
• Experience with Azure OpenAI, AWS Bedrock, Google Vertex AI, Databricks, vLLM, Triton Inference Server, MLflow, Kubeflow, AI model gateways, or similar enterprise AI platforms.
• Familiarity with Responsible AI, AI Governance, Model Risk Management, Audit Controls, AI Cost Governance, and private or open-source LLM deployments.
• Experience deploying enterprise-scale AI platforms in regulated environments.

eye