Machine Learning Safety: Evaluation Research Engineer

This role supports the design and development of safety evaluation methodologies for generative and agentic AI features that enable users across the globe to interact with our media products and services.

You will play an impactful role: shaping responsible AI and safety policies, evaluating fidelity to product safety requirements, creating risk assessments and taxonomies, curating exemplar safety evaluation datasets, and ensuring that evaluation frameworks are culturally and linguistically grounded.\nAn ideal candidate possesses a strong understanding of issues in responsible AI and A and society, technology evaluation design principles and practices, and brings experience designing evaluations to support policies and/or product requirements, classification systems, and annotation and/or study participant guidelines.

4+ years of experience in an applied research setting related to evaluation design, AI ethics, Responsible AI, AI safety, computational social science, content analysis, or a closely related field.\nStrong understanding of taxonomy design, classification systems, and annotation methodology.\nExperience developing evaluation guidelines and exemplar sets for human annotation or labeling tasks.\nDemonstrated ability to collaborate with subject matter experts (e.g., linguists, cultural consultants, multi-lingual annotators) to inform research design.\nAble to work independently to drive outcomes among cross-functional teams, with minimal direction.\nOrganized, highly attentive to detail, and manages time well.\nExcellent written and oral communication skills.\nExperience working in industry.\nAdvanced degree (MS/PhD) in Linguistics, Information Science, Computational Social Science, or a related socio-technical field.

Experience designing evaluation frameworks for multilingual or cross-cultural contexts.\nFamiliarity with responsible AI, AI safety, or content moderation policy frameworks.\nExperience with experimental design methodologies, inter-rater reliability data analysis and annotation quality assessment methods.\nPrior experience working with localization, internationalization, or language service teams.\nExperience with survey design, AI policy development, and/or structured content analysis methodologies.