Looking for AI Prompt & Data Quality Specialist
Remote • Posted 6 hours ago • Updated 6 hours ago

Xoriant Corporation
Dice Job Match Score™
🔗 Matching skills to job...
Job Details
Skills
- Artificial Intelligence
- Automated Testing
- Use Cases
- Prompt Engineering
- SAFE
- Semantics
- Extraction
- Evaluation
- Workflow
- FOCUS
- Writing
- Personas
- English
- Data Quality
- Attention To Detail
Summary
Job Title: AI Prompt & Data Quality Specialist
Location: US Remote
Duration: 6+ months
We are looking for a meticulous AI Prompt & Data Quality Specialist to lead our "Human-in-the-Loop" workflows. In this hybrid role, you will not only design the prompts that drive our AI, but you will also define the Ground Truth for what "good" looks like. You will be responsible for creating high-quality evaluation datasets, grading model outputs, and refining instructions to ensure our AI models are accurate, safe, and aligned with user needs.
- Prompt Engineering & Tuning
- Draft and develop complex system prompt for various use cases (e.g., summarization, extraction, persona-based chat).
- Implement advanced strategies (Chain-of-Thought, Few-Shot) to guide the model through complex reasoning tasks.
- Iterate on prompt versions to reduce latency and token usage while maintaining quality.
- Data Labeling & Quality Assurance (New)
- Review and Annotate: Systematically review model outputs to tag them for accuracy, tone, safety, and hallucinations.
- RLHF Support: Perform "A/B ranking" on model responses (choosing the better of two options) to support Reinforcement Learning from Human Feedback.
- Create Golden Sets: Write "ideal" human-generated responses to serve as Ground Truth for automated testing and fine-tuning.
- Maintain Guidelines: Develop and update the Annotation Rubric the detailed instructions that define exactly what counts as a "correct" or "safe" response.
Must-Haves
- Linguistic Precision: Exceptional command of English grammar and semantics. You can spot the difference between "good" and "great" writing instantly.
- Experience with LLMs: Familiarity with GPT-4, Claude, or similar models, including their limitations (hallucinations, bias).
- Extreme Attention to Detail: Ability to perform repetitive data review tasks without losing focus or consistency.
- Dice Id: xorca001
- Position Id: 8873636
- Posted 6 hours ago
Company Info
Xoriant is a Sunnyvale, CA headquartered digital engineering firm with offices in the USA, Europe, and Asia. From Tech Startups to Fortune 100 Enterprises, we enable innovation, accelerate time to market, and ensure client competitiveness across industries. Across all our focus areas – platform engineering, cloud, data & and AI, and Security – every solution we develop benefits from our product engineering DNA and culture of innovation. It also includes successful methodologies, framework components, and accelerators for rapidly solving critical client challenges. For 30 years and counting, we have taken great pride in the longlasting, deep relationships we have with our clients.
For further information about Xoriant, please visit our website
Similar Jobs
It looks like there aren't any Similar Jobs for this job yet.
Search all similar jobs
