Lead Technology Product Manager – AI Quality & Evaluation
Locations: Waltham, MA | Philadelphia, PA | Coppell, TX | Other eligible U.S. office locations
Work Arrangement: Hybrid – 8 days per month onsite
Employment Type: Full-Time
Job Summary
We are seeking a Lead Technology Product Manager specializing in AI Quality & Evaluation to translate quality standards and evaluation strategies into scalable product capabilities. This role focuses on AI evaluation workflows, automation, monitoring, quality reporting, and responsible AI practices.
The ideal candidate will have strong product management experience and the ability to collaborate with clinical, engineering, data science, compliance, and business teams.
Key Responsibilities
- Translate quality standards and evaluation methodologies into product requirements and roadmap priorities.
- Lead product discovery and delivery for AI evaluation tools, workflows, automation, and reporting capabilities.
- Define requirements for review workflows, issue triage, launch readiness, and post-launch monitoring.
- Partner with engineering, data science, clinical, legal, compliance, and customer-facing teams.
- Identify opportunities to automate evaluation processes while maintaining appropriate expert oversight.
- Use evaluation insights to improve product quality, user experience, content quality, and model behavior.
- Define metrics, dashboards, and decision criteria for measuring AI quality and performance.
- Monitor emerging practices in AI evaluation, benchmarking, and responsible AI.
- Communicate product decisions, quality outcomes, risks, and recommendations to stakeholders.
Required Qualifications
- 5+ years of product management experience in healthcare, enterprise SaaS, data-driven products, platform products, quality, or related fields.
- Strong experience in product discovery, requirements definition, roadmap planning, experimentation, and product execution.
- Ability to translate complex technical, clinical, quality, and risk concepts into practical product requirements.
- Experience working with engineering, data science, clinical, compliance, or operational teams.
- Strong analytical skills and experience defining metrics, dashboards, workflows, and decision criteria.
- Excellent communication and stakeholder management skills.
- Interest in AI technologies, evaluation methodologies, responsible AI, and emerging technology trends.
Preferred Qualifications:
- Experience with AI-enabled products, healthcare technology, data products, search/retrieval quality, or decision-support systems.
- Familiarity with AI evaluation methods, benchmarks, monitoring, automation, and human-in-the-loop review.
- Experience in healthcare, clinical workflows, regulated environments, or professional-information products.
Candidates must be comfortable with the stated hybrid work arrangement and eligible U.S. office locations.
