Distinguished AI Engineer
Location: Jersey City, NJ (Hybrid – 4 Days Onsite)
Role Purpose & Key Responsibilities
The Distinguished AI Engineer serves as the enterprise technical authority responsible for defining the architecture, engineering standards, platform strategy, and technical governance for the organization's AI Research & Innovation Platform (AIRP) and enterprise Generative AI ecosystem. This role establishes the long-term vision for scalable, secure, reusable, observable, resilient, and cost-efficient AI platforms while ensuring the current AWS-first implementation remains cloud-agnostic by design.
As a Distinguished-level technical leader, this role owns the enterprise AI platform blueprint and defines reference architectures supporting Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Agentic AI, AI Gateways, Model Hubs, AI Orchestration, AI Infrastructure, LLMOps, MLOps, Model Evaluation, AI Observability, and Enterprise AI Governance. The platform must enable AI for Business, AI for Engineering, and Responsible Citizen Development through standardized engineering practices, reusable platform capabilities, and enterprise guardrails without fragmenting governance, security, or operational controls.
The successful candidate will establish enterprise standards for Terraform Infrastructure-as-Code (IaC), DevSecOps, CI/CD pipelines, Kubernetes, GPU infrastructure, model-serving platforms, AI security, operational resilience, observability, monitoring, governance, and production operations while partnering with Engineering, Architecture, Cybersecurity, Product, Risk, Compliance, Legal, Audit, Data Governance, and Executive Leadership teams. This role provides technical assurance for the organization's highest-impact AI initiatives and evaluates emerging AI technologies based on business value, scalability, portability, regulatory compliance, and long-term architectural sustainability.
Key Responsibilities
- Define and own the target-state enterprise AI architecture for AIRP, including LLM platforms, Model Hubs, AI Gateways, Retrieval-Augmented Generation (RAG) services, Agentic AI frameworks, orchestration platforms, model-serving infrastructure, vector search, AI APIs, and enterprise AI platform capabilities.
- Establish enterprise engineering standards for AI Software Development Lifecycle (AI SDLC), LLMOps, MLOps, AI model lifecycle management, evaluation frameworks, release management, operational resilience, observability, monitoring, production support, and platform operations.
- Design and implement an AWS-first AI platform architecture that aligns with a cloud-agnostic enterprise blueprint, minimizing vendor lock-in while enabling future multi-cloud or hybrid cloud adoption.
- Define enterprise standards for Terraform modules, reusable Infrastructure-as-Code (IaC) templates, environment provisioning, CI/CD pipelines, deployment automation, approval workflows, secrets management, rollback strategies, observability, operational controls, and platform governance.
- Architect secure, scalable, resilient, and cost-optimized AI inference platforms supporting cloud-native, hybrid, private cloud, GPU-accelerated, Kubernetes-based, and containerized deployment models.
- Define enterprise guardrail frameworks addressing hallucination mitigation, bias monitoring, Responsible AI controls, prompt injection protection, AI security, data leakage prevention, harmful content detection, model governance, human oversight, and AI risk management.
- Lead architecture reviews, technical design governance, and solution assurance for high-risk, high-impact AI initiatives, ensuring compliance with enterprise architecture, cybersecurity, regulatory, risk, audit, and governance standards.
- Collaborate with Cybersecurity, Enterprise Architecture, Product Management, Engineering, Risk Management, Compliance, Legal, Audit, Data Governance, Cloud Engineering, Infrastructure, and Citizen Development Enablement teams to deliver secure, scalable enterprise AI capabilities.
- Evaluate emerging AI technologies, foundation models, LLM platforms, AI infrastructure, orchestration frameworks, vector databases, AI gateways, GPU technologies, and cloud services, recommending adoption based on business value, technical maturity, scalability, portability, operational complexity, cost optimization, security, and regulatory alignment.
- Mentor Principal Engineers, AI Architects, Platform Engineers, and Engineering Leadership by defining architectural best practices, reusable design patterns, engineering standards, and enterprise AI operating models.
- Drive enterprise-wide adoption of reusable AI platform capabilities that accelerate delivery while maintaining consistency, governance, security, reliability, and operational excellence across multiple business domains.
Must-Have Candidate Profile
- 10+ years of experience in AI/ML systems, distributed systems, enterprise architecture, cloud platform engineering, or AI platform engineering, with a proven track record of leading enterprise-scale technical initiatives.
- Deep expertise in Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), embeddings, vector databases, model serving, AI orchestration, Agentic AI, LLMOps, MLOps, AI evaluation frameworks, AI observability, and enterprise AI infrastructure.
- Proven experience defining enterprise architecture, reference architectures, technical standards, reusable platform capabilities, engineering governance, and enterprise AI platform strategies across multiple engineering organizations.
- Strong hands-on experience designing AWS cloud architectures (or equivalent hyperscaler platforms) with the ability to build cloud-agnostic AI platforms supporting hybrid and multi-cloud deployment strategies.
- Extensive experience establishing Terraform Infrastructure-as-Code (IaC) standards, DevSecOps practices, CI/CD pipelines, Kubernetes, container orchestration, secure deployment patterns, observability, resiliency strategies, and enterprise platform operating models.
- Strong understanding of AI platform security, Responsible AI, AI governance, model risk management, AI observability, production operations, enterprise technology governance, cybersecurity, compliance, and risk management.
- Exceptional leadership, communication, and stakeholder management skills with the ability to influence executive leadership, enterprise architects, engineering teams, business stakeholders, cybersecurity, compliance, legal, risk, and governance organizations without direct reporting authority.
Preferred Experience
- Experience designing enterprise AI platforms within Global Banking, Financial Services, FinTech, Insurance, Healthcare, or other highly regulated industries.
- Experience building or leading enterprise AI platforms, private LLM deployments, AI Model Hubs, AI Gateways, enterprise inference platforms, multi-cloud AI architectures, cloud-native AI platforms, or enterprise AI platform blueprints.
- Experience with AWS Bedrock, Amazon SageMaker, Kubernetes, GPU clusters, vector databases, LangChain, Semantic Kernel, LlamaIndex, MLflow, Databricks, AI observability platforms, and enterprise AI engineering ecosystems.
- Familiarity with Responsible AI, AI Governance, Model Risk Management, Technology Risk Controls, Regulatory Compliance, Privacy, Audit Requirements, Citizen Development Governance, and Enterprise AI Operating Models.
eye
Never miss an opportunity! Create an alert based on the job you applied for.