Siri is becoming a truly personal, intelligent assistant. It understands your personal context, is aware of what's on your screen, and takes action in and across your apps, all with privacy at its core. Delivering these capabilities to Apple's standard of quality requires evaluation systems that are scalable, reliable, and built for the complexity of agentic AI. Evaluation signals inform every model and product decision before a Siri release ships.
As an Engineering Program Manager for Siri Evaluation, you will lead the programs that build Siri's evaluation infrastructure, tooling, and automated evaluators. If you thrive in helping drive clarity and focus across teams and helping build the foundation for the future of Siri and Apple Intelligence sounds exciting, we'd love to hear from you!
Description
You will drive evaluation programs end to end, from requirements and system architecture through timelines, milestones, and risk management. Partnering with Engineering, Quality Engineering, Design, and Product teams, you will define the evaluation roadmap, align evaluation needs across Siri features, and scale pipelines to keep pace with model and product development.
You will make sure evaluation results are reproducible, trustworthy, and actionable, so teams can assess model quality and release readiness with confidence. You will identify gaps in coverage, throughput, and turnaround time, and lead the work to close them.
Clear communication is essential. You will provide timely, objective updates on program status, risks, critical issues and mitigations, plan changes, and requests for action. You will also bring teams together so the right discussions happen and the right decisions get made.
Minimum Qualifications
5+ years of program or project management experience in software development, machine learning, or a related field.
BS/MS in Computer Science, Engineering, or a related field.
Solid understanding of system architecture and design trade-offs, with thorough knowledge of testing, build, and release processes for complex software.
Experience driving and delivering critical project features that involve integration with multiple different teams.
Familiarity with ML development workflows, including model evaluation, quality metrics, and data pipelines.
Proven track record of leading and shipping multiple complex, cross-functional projects at the same time.
Ability to navigate ambiguity, identify the most critical/urgent work, set priorities, and defend trade-offs.
Excellent written and verbal communication, including concise status reports and issue summaries.
Articulate and facilitative in meetings, demonstrating the ability to reshape messaging for each audience including at the executive level.
Experience working cross-functionally with people at all levels of seniority & leadership.
Preferred Qualifications
Experience working with voice assistant technologies.
Experience with evaluation of large language models (LLMs) or agentic AI systems, such as benchmarks, human grading, or automated graders.
Experience scaling evaluation or testing infrastructure, including distributed pipelines, compute planning, or tooling adoption.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: 90733111
- Position Id: d261fc3e5556d7d17548023c14a52b16
- Posted 3 hours ago
