Overview
Skills
Job Details
Operations Data Analyst Site Operations_Onsite
Work Location: Onsite Burlingame (CA) or Redmond (WA)
Contract
Key Responsibilities:
- Vendor Management & Delivery
- Oversee annotation vendor performance, ensuring timely and high-quality data deliveries.
- Track and report on vendor success metrics.
- Coordinate onboarding & certification of new raters.
- Guideline Implementation & Development
- Ensure correct implementation of annotation guidelines.
- Partner with researchers to develop, refine, and update annotation guidelines.
- Quality Assurance & Data Quality
- Uphold the researcher quality bar and high data quality standards.
- Oversee DQA (Double Quality Assurance) processes and success.
- Develop, review, and improve quality assurance processes.
- Manage DQA resources and reporting.
- Training & Coordination
- Organize and deliver annotator training sessions.
- Coordinate with annotation vendors for ongoing training and calibration.
- Reporting & Communication
- Report on annotation metrics and quality trends.
- Provide monthly written updates.
- Communicate expectations, progress, and changes with research and vendor stakeholders.
Skills:
Must Haves:
3+ years experience in annotation project management roles in all 3 skill sets:
1. Training annotators on guidelines
2. Using annotation tooling
3.Quality Assurance of annotations
Proven ability to write clear annotation guidelines and define KPIs for quality measurement.
Experience training raters or annotators on guidelines and quality standards.
Strong technical aptitude to navigate annotation UIs and communicate effectively with software engineers about pipeline issues.
Excellent written and verbal communication skills, including the ability to document processes and report metrics clearly.
Experience coordinating with vendors and cross-functional teams to ensure project success.
Ability to work independently in a fast-paced, ambiguous environment.
Nice to Haves:
Background or familiarity with software engineering concepts and troubleshooting.
Education:
Bachelor s degree or higher