Hi,
The following requirement is open with our client.
Title : AI/ML Ops Engineer
Location : Sunnyvale, CA / Austin, TX
Rate : $76/hr on W2
Duration : 12 Months
Relevant Experience (in Yrs.): 15+
Detailed Job Description
We are seeking a Senior AI/ML Ops Engineer with strong experience in ML engineering, MLOps, platform engineering, or SRE. The ideal candidate will have hands-on experience with LLMs and LLM-powered applications in production, along with strong expertise in ML/LLM evaluation, observability, Python, and CI/CD.
Key Responsibilities
· Build and maintain ML/LLM evaluation systems, test harnesses, and benchmark datasets.
· Develop automated scoring and evaluation frameworks, including LLM-as-Judge approaches.
· Implement regression detection and automated quality checks for ML/LLM applications.
· Build reliable and well-tested internal platforms and tooling using Python.
· Implement monitoring and observability for ML/LLM systems.
· Work with tools such as OpenTelemetry, Datadog, Grafana, and Prometheus.
Must Have Skills
· MLOps / ML Engineering
· LLM Production Experience – 2+ years
· LLM Evaluation & Testing
· Python
· CI/CD + Observability