Engineering Program Manager, Siri and Apple Intelligence Evaluation

Apple is where individual imaginations gather together, committing to the values that lead to great work. Every new product we build, service we create, or Apple Store experience we deliver is the result of us making each other's ideas stronger. That happens because every one of us shares a belief that we can make something wonderful and share it with the world, changing lives for the better. It's the diversity of our people and their thinking that inspires the innovation that runs through everything we do. When we bring everybody in, we can do the best work of our lives. Here, you'll do more than join something - you'll add something.\\n\\nWe're looking for a Program Manager with a strong track record of building and leading effective evaluation programs demonstrating success leading via data. As a leader in the AIML Siri and Apple Intelligence evaluation team, you will lead initiatives in evaluation for foundation models and GenAI features powering Siri and Apple Intelligence experiences that are critical to Apple's future.

You will work with top tier data scientists, engineers, research teams, and product teams across Apple to help ensure we deliver high-quality, safe, and beneficial AI-powered experiences that over 1 billion customers expect and love. This role requires technical depth in evaluation methodologies combined with strong program management expertise to drive comprehensive assessment of model capabilities, safety, helpfulness, and user experience quality.

Bachelor's degree in Statistics, Business Intelligence, Computer Science, other Quantitative Sciences, or related field and equivalent experience\n8+ years of experience in driving large scale program building machine learning powered products or analytics to support product development\n5+ years of experience managing programs in AI powered product space, preferably experience in evaluation of ML/AI products\nAbility to deal with ambiguities, drive disambiguation and clarities around evaluation methodologies, shepherd multiple teams to converge on rigorous measurement frameworks\nExperience designing and implementing evaluation systems for machine learning models, particularly large language models or conversational AI systems\nProgram management skills including program structuring and managing multiple work streams interdependently across research, engineering, and product teams\nProblem-solving skills with attention to details in identifying edge cases, failure modes, and capability gaps\nAbility to communicate abstract ideas clearly, manage comprehensive yet succinct program status updates to all levels of audience, both verbally and in written forms\nProven adaptability and agility in making adjustments to program strategy and plan with evolving model capabilities and product decisions

Master's or PhD degree in Statistics, Machine Learning, Computer Science, other Quantitative Sciences, or related field and equivalent experience\nExperience with statistical analysis and drawing meaningful conclusions from large-scale evaluation datasets\nDeep understanding of LLM capabilities, limitations, and safety considerations\nSelf-sufficient in analyzing and drawing conclusions about model quality, user experience, and product opportunity from raw and refined evaluation data\nPlayer-coach capable of personally leading large evaluation initiatives while coaching team members along the way and mentoring team members to grow evaluation expertise