Generative AI Post-Doctorate Fellow

Generative AI Post Doctorate Fellow

College Board - Global Strategy and Talent

Location: This is a fully remote role. Candidates who live near CB offices have the option of being fully remote or hybrid (Tuesday and Wednesday in office).

Type: This is a full-time term position that will last for 1 year from the start date.

About the Team

College Board is taking a mission-first approach to generative AI, envisioning a future where humans drive responsible generative AI transformation within education. As the technology evolves, we're accelerating experimentation and building staff capacity with purpose and care. Our aim is simple: deliver outsized value through ethical, real-world applications that serve students, educators, and members.

The GenAI Studio is College Board's internal innovation engine behind this work. Launched in 2024, the Studio exists to accelerate experimentation and unlock the transformative potential of GenAI in service of our mission.

Spanning our Strategy and Technology divisions, we are a cross-functional, entrepreneurial team unified by a shared belief that AI, when guided by ethics and rigor, can enhance learning outcomes and operational excellence. From workforce enablement to product R&D, the Studio operates at the forefront of the GenAI landscape: enabling a culture of experimentation, building cutting-edge tools, and embedding human-centered AI innovation and literacy throughout our ecosystem. Join us in turning bold ideas into real impact for millions of students.

About the Opportunity

Our charge is to rethink assessment for an AI enabled world, test what should replace legacy patterns, and publish what we learn so the field can move forward. The fellowship blends research and invention. You will frame bold questions, design auditable experiments, and demonstrate when a new format produces stronger evidence of what students know and can do, compared with today's approaches. As our one-year Generative AI Post-Doctoral Fellow, you will step out of the purely academic sphere and into a mission-driven nonprofit that delivers assessments and learning tools to 7 million+ students each year. Working alongside researchers, psychometricians, product strategists, and engineers, you'll translate cutting-edge scholarship in learning science into field-tested AI prototypes that expand educational access, enhance engagement, and surface richer evidence of what students know and can do. You'll gain hands-on experience shaping adaptive, game-based, and non-cognitive assessments-while helping set national benchmarks for responsible, equity-centered AI in K-12 and higher-ed contexts.

In this role, you will:

Develop best practices and design principles for new AI-driven learning & assessment prototypes (33%)

• Survey and synthesize the research base on adaptive learning, gamification, and non-cognitive assessment to define evidence-backed design heuristics.
• Lead co-design sprints with psychometricians, UX researchers, and engineers to embed those heuristics in prototypes and pilots.
• Build evaluation frameworks (validity, reliability, engagement, fairness) to judge prototype performance and iterate rapidly.
• Author "Design Principles" playbooks and contribute to internal standards that guide future College Board AI initiatives.

Distill learning-science and assessment best practices into effective LLM contexts (33%)

• Translate analytic rubrics, exemplar responses, and scoring anchors into high-leverage prompt libraries, system messages, and retrieval chains for LLM-based tutors and scorers.
• Prototype few-shot and chain-of-thought prompting strategies that surface metacognitive feedback and scaffold student reasoning.
• Collaborate with engineers to automate rubric-to-prompt conversions and ensure version control across iterative assessment designs.
• Establish validation pipelines-quantitative (e.g., QWK, IRT metrics) and qualitative (e.g., think-aloud reviews)-to monitor alignment between LLM outputs and intended learning goals.

Supporting LLM prototype experiments informed by learning science/assessment best practices (33%)

• Co-design and analyze rapid LLM prototype experiments (A/B & quasi-experimental) to validate pedagogical soundness, fairness, and psychometric quality.

About You

To qualify for this role, you must have:

• PhD (completed by Aug 2025) in Learning Sciences, Educational Psychology, AIED, Measurement, Cognitive Science, or related field
• Demonstrated research on adaptive/gamified or non-cognitive assessment design
• Hands-on experience with large-language-model prompting / evaluation (OpenAI, Anthropic, etc.) or learning-analytics pipelines.
• Strong expertise in statistical modeling and data analysis using appropriate statistical methods and software (e.g., SAS, Stata, SPSS, R, Python); familiarity with IRT or ML fairness a plus.
• Experience with large, national datasets related to the education industry and the ability to work accurately and quickly with new datasets
• Experience using SQL to access data for data analysis and/or a willingness to learn
• Experience with survey design and analysis, and Qualtrics a plus
• Adept problem-solving skills, including using data to inform decisions and actions
• The ability to travel 4-6 times a year to College Board offices or on behalf of College Board business
• Authorization to work in the United States

All roles at College Board require:

• A passion for expanding educational and career opportunities and mission-driven work
• Authorization to work in the United States for any employer
• Curiosity and enthusiasm for emerging technologies, with a willingness to experiment with and adopt new AI-driven solutions and a comfort learning and applying new digital tools independently and proactively.
• Clear and concise communication skills, written and verbal
• A learner's mindset and a commitment to growth: welcoming diverse perspectives, giving and receiving timely, respectful feedback, and continuously improving through iterative learning and user input
• A drive for impact and excellence: solving complex problems, making data-informed decisions, prioritizing what matters most, and continuously improving through learning, user input, and external benchmarking.
• A collaborative and empathetic approach: working across differences, fostering trust, and contributing to a culture of shared success.

About Our Process

• Application review will begin immediately and will continue until the position is filled. This role is expected to accept applications for a minimum of 5 business days.
• While the hiring process may vary, it generally includes: resume and application submission, recruiter phone/video screen, hiring manager interview, performance exercise such as live coding, a panel interview, a conversation with leadership and reference checks.

What We Offer

At College Board, we offer more than just a paycheck-we provide a meaningful career, a supportive team, and a comprehensive package designed to help you thrive. We're a self-sustaining nonprofit that believes in fair and competitive compensation, grounded in your qualifications, experience, impact, and the market.

A Thoughtful Approach to Compensation

• The hiring range for this role is $104,000-$170,000.
• Your exact salary will depend on your location, experience, and how your background compares to others in similar roles at the College Board.
• We aim to make our best offer upfront-rooted in fairness, transparency, and market data.
• We adjust salaries by location to ensure fairness, no matter where you live.

You'll have open, transparent conversations about compensation, benefits, and what it's like to work at College Board throughout your hiring process. Check out our careers page for more.