Data Tester – Snowflake Cortex & AI Testing
Location: Boston, MA (Remote)
Contract
Summary
We are seeking a highly experienced Data Test Lead to drive quality engineering and validation across next-generation data, analytics, and AI platforms. This role is critical to ensuring the accuracy, reliability, and trustworthiness of enterprise data assets and AI-driven insights, with a strong focus on Snowflake-based ecosystems and emerging AI capabilities enabled through Snowflake Cortex.
The ideal candidate will bring deep expertise in data warehouse testing, ETL/ELT validation, BI testing, and hands-on exposure to AI/ML-driven data solutions, including LLM-based transformations, semantic search, and intelligent data pipelines. This role requires the ability to handle both deterministic data validation and non-deterministic AI output validation, ensuring compliance with enterprise quality, governance, and regulatory standards.
Key Responsibilities
1. Data Platform & Pipeline Validation
- Lead end-to-end testing of modern data platforms, including ingestion, transformation, and consumption layers.
- Validate Snowflake-based data pipelines, DBT transformations, and complex business logic implementations.
- Perform source-to-target reconciliation, ensuring completeness, accuracy, and referential integrity.
- Establish and enforce data quality rules, profiling techniques, and anomaly detection frameworks.
2. Snowflake Cortex & AI Testing
- Define and execute comprehensive testing strategies for AI-enabled data solutions built using Snowflake Cortex capabilities
- LLM-driven transformations (e.g., summarization, classification, SQL generation)
- Semantic/vector-based search pipelines
- AI agents and automated workflows
- Validate AI outputs for correctness, consistency, bias, explainability, and hallucination risks.
- Establish testing methodologies for:
- Prompt engineering validation and optimization
- Non-deterministic output validation
- Model/data drift monitoring
- Ensure AI use cases comply with data governance, security, and regulatory frameworks (e.g., GxP).
3. BI & Analytics Testing
- Lead validation of Power BI dashboards, reports, and semantic models against business KPIs and requirements.
- Validate AI-assisted analytics use cases, including natural language query-to-SQL and insight generation.
- Ensure consistency between AI-generated insights and underlying structured data.
4. Test Automation & Engineering
- Design and implement scalable, reusable test automation frameworks for:
- Data validation (SQL-based, metadata-driven)
- AI output verification and regression testing
- Structured vs semi/unstructured data validation
- Leverage Python, SQL, and data validation tools to automate complex reconciliation scenarios.
- Integrate testing into CI/CD pipelines, enabling continuous testing in modern data delivery ecosystems.
5. Test Strategy, Governance & Compliance
- Own and drive Test Strategy, Test Plans, and Test Governance for enterprise data and AI programs.
- Introduce AI-specific testing dimensions:
- Fairness, explainability, transparency
- Auditability and traceability
- Risk-based testing approaches
- Align validation approaches with enterprise data governance, lineage, and catalog frameworks.
6. Leadership & Stakeholder Management
- Collaborate with data engineers, AI/ML engineers, business analysts, and product owners to define and refine requirements.
- Lead defect management, root cause analysis, and resolution across data and AI pipelines.
- Mentor and guide QA teams on AI testing best practices and Snowflake Cortex capabilities.
- Actively participate in Agile ceremonies, sprint planning, and release governance.
Required Skills & Experience
- 8+ years of experience in Data Testing / ETL Testing / BI Testing, including leadership responsibilities.
- Strong expertise in:
- Snowflake (mandatory) – data validation, performance, transformation testing
- DBT (mandatory) – model validation, lineage, and business logic testing
- Advanced SQL – reconciliation, profiling, and anomaly detection
- Proven experience in testing AI/ML or data-driven analytics solutions, including:
- LLM output validation and quality assurance
- Non-deterministic testing approaches
- Hands-on exposure to Snowflake Cortex or similar AI-enabled platforms
- Strong experience in Power BI / BI validation and dashboard testing
- Expertise in test management and defect tracking tools (JIRA, ALM, Azure DevOps)
- Experience with automation frameworks (Python, SQL-based validation frameworks)
- Familiarity with Agile/DevOps practices (Git, CI/CD pipelines)
Preferred Qualifications
- Experience in AI-driven data applications, including:
- RAG pipelines and semantic search
- NLP and conversational analytics
- Knowledge of vector embeddings, similarity search, and AI data validation techniques
- Exposure to cloud ecosystems (AWS, Azure, Google Cloud Platform) for data and AI workloads
- Experience in Life Sciences / Pharma domain (GxP compliance, validation practices)
- Understanding of AI ethics, bias detection, and explainable AI concepts
- Prior experience leading data modernization or AI transformation programs
Never miss an opportunity! Create an alert based on the job you applied for.
