ML / GenAI Ops Engineer

ML/GenAI Ops Engineer

Long term Contract

Remote

About the Role

We are seeking a GenAI Ops Engineer to transform prototypes into production-ready platforms. The role ensures internal GenAI tools are enterprise-grade while also being encapsulated and infra-agnostic for safe extension to customer environments. You will design the pipelines, infrastructure, and cost-optimization strategies that make GenAI scalable, reliable, and compliant across both internal operations and external customer deployments.

Responsibilities

1. Internal Platformization

• Transition PoCs/MVPs into scalable, reliable production systems.
• Build CI/CD pipelines with version control for prompts, models, and workflows.
• Develop reusable internal GenAI frameworks that reduce time-to-deploy across teams.
• Implement observability pipelines for latency, token usage, drift, and hallucination monitoring.
• Apply FinOps practices to control infrastructure and LLM API spend.

2. Customer Deployment Enablement

• Create encapsulated, infra-ready GenAI modules that can be deployed in cloud, hybrid, or on-prem customer environments.
• Ensure multi-tenant readiness and infrastructure-agnostic design for customer-facing solutions.
• Optimize performance, governance, and compliance for enterprise customer standards.
• Provide deployment playbooks, templates, and monitoring dashboards for customer success teams.
• Continuously evaluate open-source vs. API LLM models for cost-performance tradeoffs in customer delivery.

Requirements

• Proven experience in DevOps/MLOps, with specialization in GenAI/LLM workloads.
• Strong expertise in cloud platforms (AWS, Azure, Google Cloud Platform) and IaC (Terraform, Pulumi).
• Skilled in containerization & orchestration (Kubernetes, Docker).
• Familiarity with LangChain, LangGraph, RAG patterns, vector DBs.
• Experience in infrastructure + LLM API cost optimization.
• Strong scripting and automation background (Python, Bash, SQL).

Nice to Have

• Hands-on deployment of open-source LLMs in production (Mistral, LLaMA, Phi, Qwen).
• Experience with multi-tenant SaaS infra and customer-specific deployments.
• Knowledge of governance & compliance frameworks for AI/LLM in regulated industries.

What Success Looks Like

• Internal: GenAI MVPs become reusable, monitored, cost-optimized platforms across the org.
• Customer: Encapsulated, infra-ready GenAI solutions are easily deployable in diverse environments.
• Overall: Deployment is seamless, observable, and compliant, enabling both internal efficiency and external trust.