Job DescriptionAs the Principal AI/ML Engineer - Post-Deployment Governance within AI Validation & Monitoring (AVM), you will serve as the enterprise technical and methodological authority for post-deployment monitoring and reporting, measurement, lifecycle evidence, and Post Deployment Monitoring (PDM) and Post Deployment Reporting Summary (PDRS) governance. You will define risk-proportionate AIA Governance requirements and standards for monitoring readiness; performance and functionality; patient safety; adoption and fidelity; outcomes; change and retesting; metrics, formulas, baselines, targets, and thresholds; subgroup interpretation; uncertainty; and evidence confidence. You will apply data science, AI/ML engineering, statistical, and systems expertise to determine whether evidence is traceable, appropriately interpreted, proportionate to risk, and decision-ready.
Within AIA Governance, you will review drafted monitoring, reporting, measurement, and PDRS content; direct corrections and alternate approaches; consult on complex cases; establish precedent; and escalate unresolved technical or policy issues.
- Provide strategic and technical leadership for enterprise post-deployment governance, measurement, monitoring and reporting, and PDM and PDRS standards.
- Define risk-proportionate requirements across pilot, full implementation, post-deployment change, recurring PDRS, and legacy-product pathways.
- Establish standards for signals, metrics, formulas, baselines, targets, thresholds, uncertainty, evidence confidence, outcomes, and subgroup interpretation.
- Define monitoring-readiness expectations for sources, owners, collection methods, cadence, versions, limitations, lineage, Data Cards, Model Cards, handoffs, and sustainable ownership.
- Provide authoritative SME review of Governance Operations Product Lead assessment content and evidence for policy alignment, sufficiency, traceability, methodological adequacy, and decision readiness.
- Apply data science, statistical, AI/ML engineering, and systems methods to assess metric validity, source fitness, threshold logic, analyses, limitations, and conclusions.
- Review observability, logging, telemetry, workflow signals, version context, change detection, and monitoring and reporting continuity through significant changes.
- Own complex or precedent-setting questions involving monitoring, thresholds, evidence insufficiency, vendor limitations, significant change, revalidation continuity, lifecycle action, PDRS, or CAIO escalation.
- Recommend corrections, alternate methods, interim controls, additional evidence, action plans, re-review, retesting, or revalidation.
- Set precedent, issue final AVM direction, and escalate policy, clinical, cross-domain, or enterprise impasses.
- Lead PDRS templates and rubrics, evidence-confidence and escalation methods, metric libraries, executive presentation standards, and governance acceptance criteria.
- Convert recurring gaps into policy, playbooks, standard findings, rubrics, examples, training, calibration, and Product Lead enablement.
- Define enterprise requirements for TRex workflows, evidence objects, traceability, dashboards, portfolio visibility, and reusable governance capabilities.
- Coordinate with product teams, vendors, platforms, legal, committees, and enterprise groups on methods, tooling, specifications, and ownership.
- Provide clear complex-case findings that communicate limitations, confidence, required actions, and escalation triggers to technical and non-technical audiences.
- Mentor and calibrate engineers, analysts, and Product Leads; foster consistent methods and cross-lane coordination with Validation & Evaluation.
- Support audit sampling, quality assurance, enterprise learning, and continuous improvement while preserving AVM's review-and-consultation boundary.
- Provide mentorship, guidance, and technical leadership to junior engineers. May have supervisory responsibilities.
Qualifications - A master's degree in engineering, computer science, mathematics, health science, or a related field with 7 years of relevant experience, or a bachelor's degree with 9 years of relevant experience.
- Extensive (7+ years) experience applying AI and machine learning in production healthcare environments or similar highly regulated or technology focused industries, showcasing an acute understanding of healthcare technology.
- Demonstrated leadership in managing complex projects, with a proven ability to navigate intricate project requirements and deliver successful outcomes
- Proven success in fostering collaboration across diverse teams and effectively communicating complex technical concepts to non-technical stakeholders.
- Demonstrated expertise in cloud infrastructure environment and software development tools.
- Experience working with large, complex, and heterogeneous data sets, preferably in healthcare.
- Strong skills in AI/ML techniques and frameworks.
- Expertise with best practices in data engineering, data science, AI Engineering, and the MLOps communities.
- In-depth knowledge of healthcare domain, including clinical workflows, electronic health records, medical terminologies, regulatory requirements, and industry standards.
- Demonstrated leadership in administration, education, software development, and technical reporting.
- Experience mentoring and training less-experienced team members, coupled with strong interpersonal, communication, and time management skills.
Preferred Qualifications: - A Ph.D. or other doctorate is preferred.
- Experience with healthcare industry informatics standards, best practices, and common data models. Participation in national or international standards organizations or other domain-specific professional organizations, or extensive implementation experience with common data, development, and deployment standards.
- Excellent communication, collaboration, and stakeholder management skills, with the ability to effectively engage with diverse stakeholders and translate complex technical concepts and results to non-technical audiences.
- Demonstrated experience leading technical/quantitative teams in a regulated environment.
- Familiarity with systems or quality engineering best practices, regulatory standards, and compliance frameworks, with the ability to adapt these effectively to different project scenarios.
- Demonstrated experience creating risk management files and verification/validation strategies for digital health technology products within the healthcare industry.
- Demonstrated expertise in user-centered design, human factors engineering, usability testing methodologies, and evaluation across AI product development. Ability to lead expert reviews using established usability practices and methods. Presents findings in easy-to-understand terms for the business or clinical practice.
- Strong problem-solving abilities, critical thinking skills, and a passion for driving innovation and positive change in healthcare through AI technology.
- Demonstrated hands-on leadership using the TRex assessment application to govern AI tools deployed in EPIC, ANIMATE, and comparable clinical environments, including post-deployment standards, metric thresholds, evidence confidence, significant-change review, revalidation, and executive escalation.
About UsWhy Mayo ClinicMayo Clinic is top-ranked in more specialties than any other care provider according to U.S. News & World Report. As we work together to put the needs of the patient first, we are also dedicated to our employees, investing in competitive compensation and comprehensive benefit plans - to take care of you and your family, now and in the future. And with continuing education and advancement opportunities at every turn, you can build a long, successful career with Mayo Clinic.
Benefits Highlights - Medical: Multiple plan options.
- Dental: Delta Dental or reimbursement account for flexible coverage.
- Vision: Affordable plan with national network.
- Pre-Tax Savings: HSA and FSAs for eligible expenses.
- Retirement: Competitive retirement package to secure your future.
About the TeamJust as our reputation has spread beyond our Minnesota roots, so have our locations. Today, our employees are located at our three major campuses in Phoenix/Scottsdale, Arizona, Jacksonville, Florida, Rochester, Minnesota, and at Mayo Clinic Health System campuses throughout Midwestern communities, and at our international locations. Each Mayo Clinic location is a special place where our employees thrive in both their work and personal lives. Learn more about what each unique Mayo Clinic campus has to offer, and where your best fit is.
Equal Opportunity All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender identity, sexual orientation, national origin, protected veteran status or disability status. Learn more about the "EOE is the Law". Mayo Clinic participates in E-Verify and may provide the Social Security Administration and, if necessary, the Department of Homeland Security with information from each new employee's Form I-9 to confirm work authorization.
Never miss an opportunity! Create an alert based on the job you applied for.