AI Safety Review (Remote, Contract)
This Jobot Consulting Job is hosted by: Augie Ullmann
Are you a fit? Easy Apply now by clicking the "Apply Now" button and sending us your resume.
Salary: $75 - $90 per hour
A bit about us:
We are partnered with a fast growing AI data services company that builds the training and evaluation data behind many of the world's leading AI labs and foundation model developers. They are assembling a small panel of nuclear and radiological professionals to audit how these models respond to sensitive questions in the field. The panel decides where a model should answer, where it should decline, and what a correct answer looks like. That judgment carries directly into how the models behave for every reactor engineer, health physicist, regulator and student who asks them a nuclear question. They operate remotely across time zones and build these panels around people who are actively practicing in the discipline.
Why join us?
Selective Panel: A small review group rather than an open annotation pool
Fully Remote, Flexible Hours: Set your own schedule and volume across time zones
Keep Your Current Role: Structured as supplemental work. No requirement to leave your position
Fix Over-Refusal, Not Just Misuse: Legitimate professional questions that models wrongly decline are explicitly in scope and count as much as the dangerous ones
Your Judgment Sets the Standard: You write the answer the model should have given
Work Inside Your Expertise: Technical audit in your own discipline, not general purpose data work
Broad Eligibility: Open across the US, UK, Canada, Australia, New Zealand and the EU
Job Details
Key Responsibilities and Duties
Review user requests in the nuclear and radiological domain and classify each as benign, dual use or adversarial
Assess the responses three different models gave to the same request, marking each as safe or unsafe and documenting the reasoning
Rank the three responses against one another on accuracy, safety framing and usefulness
Write the reference answer the model should have given
For dual use requests, document what makes the request sensitive, what legitimate purpose it serves, and who should be able to receive an answer
Identify legitimate professional questions that models wrongly refuse, which carry the same weight as identifying dangerous content
Detect incorrect physics, wrong units or activity figures, misleading risk statements and unsupported technical conclusions
Apply project rubrics consistently and provide clear written rationale for every judgment
Escalate ambiguous or high risk cases according to project guidelines
Qualifications Needed
Relevant degree in Nuclear Engineering, Physics, Health Physics, Radiation Science, Radiochemistry, Chemistry or a related field
Minimum 5 years of professional experience in nuclear engineering, reactor physics, health physics, radiation safety, safeguards, nonproliferation, radiochemistry or nuclear regulatory work
Background in one or more of the following: nuclear engineering and reactor physics, safeguards and nonproliferation including IAEA, NRC or equivalent and export control, health physics and radiation safety, radiochemistry, or military and government nuclear service
Experience reviewing technical work against a standard, such as licensing, inspection or peer review, is a strong plus
Prior AI red teaming, technical QA, policy review or rubric based evaluation experience is a strong plus
Proven ability to judge whether technical content is accurate, safe and appropriately scoped, and to explain that judgment in precise written English
Strong written English sufficient to produce detailed rationale and reference answers
Reliable and self directed, able to deliver consistent quality in a remote contractor workflow
or resident of the United States, United Kingdom, Canada, Australia, New Zealand or the European Union
Interested in hearing more? Easy Apply now by clicking the "Apply Now" button.
Jobot is an Equal Opportunity Employer. We provide an inclusive work environment that celebrates diversity and all qualified candidates receive consideration for employment without regard to race, color, sex, sexual orientation, gender identity, religion, national origin, age (40 and over), disability, military status, genetic information or any other basis protected by applicable federal, state, or local laws. Jobot also prohibits harassment of applicants or employees based on any of these protected categories. It is Jobot's policy to comply with all applicable federal, state and local laws respecting consideration of unemployment status in making hiring decisions.
Sometimes Jobot is required to perform background checks with your authorization. Jobot will consider qualified candidates with criminal histories in a manner consistent with any applicable federal, state, or local law regarding criminal backgrounds, including but not limited to the Los Angeles Fair Chance Initiative for Hiring and the San Francisco Fair Chance Ordinance.
Information collected and processed as part of your Jobot candidate profile, and any job applications, resumes, or other information you choose to submit is subject to Jobot's Privacy Policy, as well as the Jobot California Worker Privacy Notice and Jobot Notice Regarding Automated Employment Decision Tools which are available at jobot.com/legal.
By applying for this job, you agree to receive calls, AI-generated calls, text messages, or emails from Jobot, and/or its agents and contracted partners. Frequency varies for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You can reply STOP to cancel and HELP for help. You can access our privacy policy here: jobot.com/privacy-policy
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: 91113390
- Position Id: 1952616511
- Posted 3 hours ago
Never miss an opportunity! Create an alert based on the job you applied for.