AI QA Engineer

Skills –

Quality LLM, OpenAI, Azure, Python RAG Pipeline, AgenticAI. 

Looking for a AI Quality Engineering Lead with expertise in LLMs (OpenAI, Azure OpenAI), Agentic AI, RAG pipelines, Python automation, and AI validation frameworks, responsible for driving AI quality assurance, workflow optimization, intelligent test automation, runtime reliability, and scalable enterprise AI governance across healthcare-focused AI solutions. 

Mandatory skills

• AI Validation, Runtime Assurance & Automation Support AI validation activities, including prompt testing, workflow testing, regression testing, runtime quality assurance, and production reliability support. Partner with AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, and Data teams to support scalable AI Quality Engineering and workflow automation processes across enterprise AI initiatives.
• Design and support runtime quality practices, including telemetry alignment, monitoring coordination, validation processes, and runtime reliability improvement efforts.
• Drive adoption of AI-assisted testing approaches, intelligent automation, reusable testing accelerators, and orchestration-aware testing practices.
• Support observability and runtime visibility initiatives improving reliability, traceability, and confidence across AI-enabled systems.
• Collaborate with Clinical, Operational, and Engineering stakeholders to support validation of healthcare workflows, operational processes, and AI-enabled business solutions.
• Technical Enablement, Delivery & Operational Support delivery coordination activities across AI Quality Engineering and workflow optimization initiatives, including implementation planning, issue tracking, operational support, and release coordination activities.
• Partner with stakeholders to evaluate implementation readiness, workflow dependencies, operational risks, automation opportunities, and quality considerations for AI initiatives. Support tooling evaluations, automation frameworks, orchestration tooling, and modernization initiatives supporting AI Quality Engineering maturity.
• Help establish reusable workflow automation patterns, scalable testing assets, and engineering enablement practices across delivery teams. Support adoption of modern AI Quality Engineering and workflow optimization practices across engineering and business organizations.
• Leadership, Collaboration & Continuous Improvement Lead and mentor engineers, analysts, contractors, and delivery teams while fostering a collaborative, continuously learning, and engineering-focused culture.
• Communicate implementation risks, workflow optimization opportunities, technical trade-offs, and operational recommendations to technical and business stakeholders. Promote engineering discipline, continuous improvement, responsible AI adoption, and operational accountability across AI Quality Engineering initiatives.