Role: AI Quality Engineering Lead
Location: Eden Prairie, MN, IL Chicago, Irving, TX
Work Mode: Hybrid
Experience: Minimum 8+ Years
We are seeking an experienced AI Quality Engineering Lead to support enterprise AI initiatives focused on healthcare technology. This role involves leading quality engineering practices for AI-powered applications, Large Language Models, intelligent automation, and workflow optimization while helping build scalable, reliable, and well-governed AI solutions.
Responsibilities
Lead the design and implementation of quality engineering solutions for AI-powered applications and enterprise AI platforms.
Develop scalable validation frameworks for Large Language Models, Retrieval Augmented Generation pipelines, intelligent automation, and AI-enabled workflows.
Design and implement automated testing strategies, validation processes, runtime quality controls, and reusable quality engineering frameworks.
Support modernization of quality engineering practices through automation, workflow orchestration, and AI-assisted testing.
Develop quality metrics, anomaly detection processes, validation pipelines, and governance-aligned testing approaches.
Design and improve human-in-the-loop validation workflows and operational quality processes.
Collaborate with engineering, product, business, clinical, and data teams to identify opportunities for quality improvements and workflow optimization.
Support runtime monitoring, observability, telemetry, and production reliability initiatives for AI solutions.
Participate in implementation planning, release coordination, issue tracking, and operational support for AI delivery initiatives.
Evaluate automation frameworks, testing tools, orchestration platforms, and engineering enablement practices.
Mentor engineering teams and promote quality engineering best practices across AI initiatives.
Communicate technical recommendations, implementation considerations, and quality improvements to technical and business stakeholders.
Required Skills
Minimum 8+ years of professional experience in Quality Engineering, Software Testing, or Automation.
Hands-on experience with Large Language Models including OpenAI and Azure OpenAI.
Experience working with Retrieval Augmented Generation pipelines.
Knowledge of Agentic AI concepts and intelligent automation workflows.
Strong Python programming experience.
Experience developing AI validation frameworks and automated testing solutions.
Experience with prompt validation, regression testing, runtime quality assurance, and production monitoring.
Knowledge of AI observability, telemetry, monitoring, and runtime reliability.
Experience with automation frameworks and workflow orchestration.
Strong understanding of enterprise AI governance and responsible AI practices.
Experience collaborating with cross-functional engineering and business teams.
Strong analytical, communication, and problem-solving skills.
Preferred Skills
Experience supporting healthcare technology or healthcare AI initiatives.
Knowledge of AIOps, LLMOps, AI governance, and AI operations.
Experience with enterprise AI platforms and cloud-based AI services.
Experience building reusable automation frameworks and testing accelerators.
Education
Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, Data Science, or a related field. Relevant technical certifications are a plus.
Never miss an opportunity! Create an alert based on the job you applied for.