Technology environment
Python, SQL, Azure Foundry/Azure OpenAI and other LLM provider APIs, Azure AI Search, Databricks, Snowflake, AKS and Azure App Service. Supporting dependencies may include Redis, Couchbase, Oracle and Azure Blob. Git, CI/CD and observability tools support release and operational workflows. Application-specific gateways, model frameworks and configurations are learned during onboarding.

Key responsibilities

• Provide Level 2/Level 3 support through ServiceNow. Assess business impact, diagnose incidents, coordinate recovery and escalation, and document root causes, workarounds and corrective actions.
• Troubleshoot LLM gateways and provider integrations, including authentication, quotas, rate limits, routing, timeouts, retries and approved fallback behavior. Trace failures across APIs and downstream services.
• Operate retrieval-augmented generation (RAG) and search workflows. Investigate indexing failures, stale knowledge, embeddings, filtering and relevance issues in Azure AI Search and connected data pipelines.
• Support Databricks model, processing and evaluation jobs and batch or online inference. Investigate runtime failures, dependencies, schema changes, data freshness, scoring errors and degraded predictions with data and model owners.
• Maintain evaluation datasets and automated checks for retrieval relevance, groundedness, response safety and model quality. Monitor drift and regressions using suitable baselines and business acceptance criteria; coordinate retraining or model changes with accountable owners.
• Monitor availability, latency, errors, throughput, token usage/cost, job success and output freshness. Correlate logs, metrics and traces to distinguish data, model, provider, application and infrastructure failures.
• Version code, prompts, configurations and model artifacts. Maintain reproducible environments, automated tests and controlled release, promotion and rollback procedures using Git and CI/CD.
• Support vision, image-processing or OCR workflows when assigned. Diagnose input and output-quality issues and engage domain specialists for deeper model or algorithm changes.
• Automate repeatable checks and recovery tasks. Maintain runbooks, dependency maps, handovers and escalation paths; apply approved access controls, secret handling, sensitive-data protection and AI safety practices.

• Demonstrated production support or engineering experience across both generative AI services and machine learning pipelines, including incident response, deployments, monitoring and recovery. Be able to explain direct contributions and operational outcomes in each area.
• Hands-on enterprise LLM API integration and gateway troubleshooting, plus practical RAG, embeddings, vector or hybrid search, retrieval filtering and response evaluation experience.
• Hands-on Databricks or comparable ML-platform experience, including job diagnostics, model/artifact versioning, reproducibility, batch or online inference, evaluation and rollback.
• Strong Python and SQL skills for debugging, data investigation and automation. Experience diagnosing REST APIs, authentication and service-to-service failures.
• Experience with Git, CI/CD, automated testing and controlled releases. Working knowledge of containers, Kubernetes application diagnostics and cloud service dependencies.
• Ability to select useful monitoring and evaluation measures, distinguish data problems from model regressions, and communicate business impact and technical findings clearly.
• Ability to prioritize a shared incident and improvement backlog, document operational procedures and train backup resources. A relevant degree or equivalent professional experience.

Preferred experience

Scope and working relationships

• Flexible remote and hybrid work opportunities
• Continuous learning and professional development programs
• A culture that prioritizes employee wellbeing