Job Title: Overwatch — Observability and Evaluation Engineer
Location: Charlotte, North Carolina
Job Type: Contract
Work Model: Onsite
Experience: 10+ years in Software Engineering; 3+ years in AI/ML and Machine Learning Model Operations
Job Overview
Our client is seeking a skilled ML Ops Engineer to join their team in Charlotte, NC. In this role, you will be responsible for building, deploying, and monitoring machine learning pipelines while ensuring model governance, observability, and evaluation at scale. You will work closely with cross-functional engineering teams to support high-performance AI/ML systems in production.
Key Responsibilities
- Develop and maintain ML pipelines using tools such as MLflow, Kubeflow, or Vertex AI
- Automate model training, testing, deployment, and monitoring across cloud environments (Google Cloud Platform, AWS, Azure)
- Implement CI/CD workflows for model lifecycle management including versioning, monitoring, and retraining
- Monitor model performance using observability tools and ensure compliance with model governance frameworks (MRM, documentation, explainability)
- Provision containerized environments and support model scoring via low-latency APIs
- Leverage AutoML tools (Vertex AI AutoML, H2O Driverless AI) for rapid, low-code model development and deployment
- Implement telemetry, traces, dashboards, SLOs, evaluation suites, and readiness evidence for priority agent releases
Required Skills
- Python, Java, SQL
- ML libraries: scikit-learn, XGBoost, TensorFlow, PyTorch
- MLflow, Kubeflow, Vertex AI
- Cloud platforms: Google Cloud Platform, AWS, Azure
- CI/CD, containerization, low-latency API integration
- LLM and agent evaluation, tracing, telemetry, metrics, and dashboards
- SLOs, test automation, prompt and model performance analysis
Preferred Skills
- Experience with AutoML platforms (Vertex AI AutoML, H2O Driverless AI)
- Background in model governance and MRM frameworks
- Hands-on production operations experience for AI/ML systems
Location & Work Model
Onsite in Charlotte, North Carolina. Local candidates only — in-person interviews required.
Engagement Details
Contract engagement. Start date to be confirmed. Apply with your updated resume to be considered.
Never miss an opportunity! Create an alert based on the job you applied for.
