Are you interested in working with the World s leading AI-powered Quality Engineering Company? Ready to advance your career, team up with global thought leaders across industries and make a difference every day? Join us at Qualitest!

We are looking for an AI Data Services Solution Architect to join our growing team in United States!

AI Data Services Solution Architect (Human-in-the-Loop)

Role Overview:
The AI Data Services Solution Architect supports the Sales and Delivery teams by designing data-focused AI solutions for customers. This role combines client engagement with practical expertise in data sourcing, annotation workflows, model evaluation, safety, and humanintheloop (HITL) services. The architect leads discovery discussions, defines solution patterns, supports proposals and RFPs, and ensures technical clarity for customer-facing programs.

Key Responsibilities:

Lead customer discovery sessions to understand requirements across data sourcing, annotation, evaluation, and HITL review processes.
Design endtoend solution architectures including data pipelines, annotation workflows, QA processes, and evaluation frameworks.
Define data governance considerations provenance, labeling guidelines, redaction, PII handling, and compliance alignment.
Develop HITL workflows: task routing, reviewer processes, sampling methods, gold sets, and quality controls.
Create evaluation approaches for LLMs/ML models, including metrics for grounding accuracy, hallucination checks, and safety compliance.
Recommend safety guardrails for privacy, toxicity, bias, and content compliance.
Contribute API integration patterns for data ingestion, annotation platforms, and evaluation tools; support basic UI prototypes.
Support Sales with solution documentation, diagrams, estimates, SOW inputs, and presentation materials.
Partner with Delivery for technical handoff, success criteria, and readiness planning.

Required Skills & Experience:

6 - 10+ years in solution architecture, data engineering, ML, or related technical roles with client-facing experience.
Strong understanding of:
- Data pipelines (Spark/Databricks, ADF/Glue, SQL/Python).
- Annotation/HITL platforms (Label Studio, SageMaker Ground Truth, Scale AI, Appen, etc.).
- LLM/ML evaluation concepts and safety best practices.
- API integrations and modern front-end frameworks (React/Next.js).
- Ability to communicate complex topics to both technical and business stakeholders.
- Solid documentation and diagramming skills (Visio, Lucidchart, Miro).

Must Have:

Execute API Integration.
Understanding of AI models and evaluation.
Sales experience.

Preferred Qualifications:

Experience integrating CRM, support systems, or enterprise content repositories.
Familiarity with Azure OpenAI, AWS Bedrock/SageMaker, or Vertex AI.
Knowledge of Responsible AI, data privacy, and compliance frameworks.
Success Indicators.
High-quality client discovery and clear solution documentation.
Strong POC and pilot scoping with measurable success criteria.
Annotation quality metrics (sampling accuracy, agreement rates) aligned to expectations.
Effective evaluation frameworks reducing hallucinations and improving model quality.
Smooth coordination between Sales and Delivery teams.
Location: West Coast Preferred.

AI Data Services Solution Architect

Similar Jobs