Data / AI QE Lead Retail eCommerce

Data / AI QE Lead — Retail eCommerce

Location-San Ramon, California or Beverly Hills, CA (5 Days Onsite)

Role Summary

The Data / AI QE Lead will define the quality engineering strategy for data pipelines, machine learning models, and AI-powered features across the retail eCommerce platform. This role bridges traditional data quality assurance and emerging AI/ML validation disciplines, ensuring that customer-facing capabilities — including product recommendations, personalization, search relevance, demand forecasting, and pricing intelligence — perform accurately, fairly, and reliably at scale.

Data Quality Engineering

•      Define and own the QE strategy for data assets including customer, product, inventory, transaction, and behavioral event data

•      Design and implement data validation frameworks covering completeness, accuracy, consistency, timeliness, and referential integrity

•      Lead testing of ETL/ELT pipelines, data lake and warehouse layers (raw, curated, consumption), and real-time streaming pipelines

•      Establish data contract testing practices between producing and consuming systems

•      Build automated data quality monitors and alerting that operate continuously in production environments

AI / ML Model Quality & Validation

•      Lead quality validation for ML models powering eCommerce capabilities: product recommendations, personalized search, dynamic pricing, demand forecasting, propensity models, and generative AI features

•      Define model evaluation frameworks including offline metrics and online business metrics (CTR, conversion rate, AOV, revenue lift)

•      Design and execute A/B and shadow testing strategies to validate model performance before and during production rollout

•

eCommerce Platform Integration Testing

•      Drive end-to-end quality of data flows from customer interaction events through to AI feature delivery on site, app, and email channels

•      Test integrations between the eCommerce platform and downstream data consumers including CDP, CRM, marketing automation, and analytics tools

•      Validate real-time personalization pipelines for homepage, PDP, cart, and post-purchase experiences

•      Ensure data quality for key eCommerce events: product views, add-to-cart, checkout, order confirmation, returns, and search queries

•      Test search and browse relevance improvements driven by ML rankers and query understanding models

Test Automation & Observability

•      Build and scale automated data and AI testing frameworks integrated into CI/CD and model deployment pipelines

•      Define and enforce data quality SLAs and embed automated gates into pipeline orchestration (Airflow, dbt, Spark, etc.)

•      Implement observability tooling for data pipelines and AI model inputs/outputs in collaboration with data and ML engineering

•      Drive adoption of synthetic data and data masking strategies to support safe, representative testing environments

•      Establish version-controlled, repeatable test datasets for regression testing of ML models across release cycles

Qualifications

Required

•      7+ years in data or quality engineering, with at least 2 years leading a team or technical discipline

•      Proven experience testing data pipelines (batch and streaming) across modern data stack technologies (Spark, Kafka, Airflow, dbt, Snowflake, BigQuery, Databricks, or similar)

•      Hands-on experience with ML model evaluation techniques, including offline metrics and online experimentation (A/B testing)

•      Strong SQL skills and proficiency in Python for data validation scripting and test automation

•      Familiarity with eCommerce data domains: customer behavior, product catalog, order management, inventory, and digital marketing

•      Excellent ability to communicate data and AI quality concepts to technical and non-technical stakeholders