Location: Portland, OR
Salary: $35.00 USD Hourly - $45.00 USD Hourly
Description: Our client is currently seeking a Data Engineer (I)
Position Overview
We are seeking an experienced Data Engineer to join an advanced research team dedicated to developing and refining next-generation, large-scale multimodal AI models. In this high-priority role, you will bridge the gap between human evaluation workflows and robust data pipeline development.
You will work directly with research scientists and software engineers to design human studies, manage data annotation processes, build evaluation interfaces, and create automated pipelines that directly measure and improve model performance.
Key Responsibilities
1. Data Collection & Human Evaluation (~50% Effort)
- Study & Workflow Design: Design, deploy, and manage human evaluation studies and annotation workflows to assess multimodal model behaviors.
- Interface & Tooling Development: Build and maintain front-end annotation UIs and custom tooling for human raters.
- Quality Operations & Rater Management: Oversee day-to-day operations with internal and external vendor rating teams across multiple time zones. Monitor evaluation quality, establish feedback loops, and deliver quality control checks.
2. Pipeline Automation & Analytics Engineering (~50% Effort)
- Data Pipeline Construction: Architect and maintain efficient data processing pipelines, data visualization tools, and performance analysis frameworks.
- Workflow Automation: Drive efforts to eliminate manual and repetitive data tasks, aiming for 80%-90% automation in data allocation and ingestion workflows.
- Model-Data Analytics: Develop diagnostic tools and visual dashboards to analyze how dataset changes impact model behavior across various scenarios.
Required Qualifications
- Core Software Engineering: Advanced proficiency in Python with expertise in data science and manipulation libraries (e.g., Pandas, NumPy).
- Data Pipeline Engineering: Proven hands-on track record in building, automating, and deploying production-grade data pipelines.
- Human-in-the-Loop Workflows: Demonstrable experience building annotation UIs and designing human study/evaluation frameworks.
Preferred Qualifications
- Generative AI Familiarity: Working knowledge of large AI model evaluation, multimodal datasets, and model behavior diagnostic methods.
- Cloud & Modern Tooling: Familiarity with cloud platforms and modern workflow automation or AI agent frameworks.
- Academic Background: An advanced degree (Master's or Ph.D.) in Computer Science, Data Science, or a related quantitative field is a plus, though practical engineering capability is prioritized.
Work Environment & Location Requirement
- Location Restriction: Candidates must reside and work remotely from Oregon.
- Collaboration: High-touch collaboration environment working alongside researchers and engineers. Daily code reviews and frequent communication with globally distributed annotation teams are expected.
Medical, dental, and vision insurance are available to qualified candidates who meet eligibility requirements.
By providing your phone number, you consent to: (1) receive automated text messages and calls from the Judge Group, Inc. and its affiliates (collectively "Judge") to such phone number regarding job opportunities, your job application, and for other related purposes. Message & data rates apply and message frequency may vary. Consistent with Judge's Privacy Policy, information obtained from your consent will not be shared with third parties for marketing/promotional purposes. Reply STOP to opt out of receiving telephone calls and text messages from Judge and HELP for help.
Contact: This job and many more are available through The Judge Group. Please apply with us today! 
Never miss an opportunity! Create an alert based on the job you applied for.