AI / ML Architect testing / Validation

Job Description: AI / ML Architect testing / Validation

Location Hybrid - Philadelphia - PA

Experience: 12+ Years

Job Summary :

As an Architect with expertise in AI/ML Validation and Testing, you will play a pivotal role in designing and implementing robust data solutions. You will leverage your skills in AWS technologies, including Redshift, S3, and Glue, to optimize data workflows. Collaborating with cross-functional teams, you will ensure seamless integration and automation of processes, enhancing the company s data-driven decision-making capabilities.

Required Skills : Technical Skills: AI/ML Validation/Testing, Amazon Redshift,AWS S3,AWS Glue ETL,AWS Glue, Jira, Test Automation, Python, ETL Validation

Key Responsibilities

- Define, implement, and maintain automated test suites to validate data pipelines, transformations, and analytical outputs.

- Collaborate with data engineers, modelers, and AI engineers to establish and enforce data quality standards and best practices.

- Conduct functional, regression, and integration testing of ETL jobs, data ingestion pipelines, and model outputs.

- Develop test strategies to validate structured (OLTP,OLAP) and unstructured data (logs, PDFs, etc.).

- Build automated tools and dashboards for data profiling, data drift detection, and quality scorecards.

- Identify root causes of data quality issues and coordinate remediation efforts with engineering teams.

- Validate data accuracy across Redshift, Postgres, and data marts using SQL and scripting tools.

- Ensure traceability from business rules and requirements to test cases and final outputs.

- Participate in Agile/Scrum processes including sprint planning, demos, and retrospectives. Required Qualifications

- 5+ years of experience in QA or Quality Engineering roles in data-driven environments.

- Hands-on experience with test automation frameworks (e.g., PyTest, Great Expectations, dbt tests).

- Proficient in SQL for validating data transformations across platforms like Redshift, Postgres, or Snowflake.

- Familiarity with ETL testing, data ingestion pipelines, and data quality validation techniques.

- Experience testing and validating BI dashboards and reports, preferably using tools like Tableau.

- Exposure to CI/CD practices and integration of tests into deployment pipelines.

- Ability to read and write test cases in Python or other scripting languages.

- Experience with version control tools (e.g., Git) and JIRA or similar test tracking systems.

Preferred Qualifications

- Experience testing unstructured data pipelines, including logs, PDF documents, or application telemetry.

- Familiarity with data profiling and anomaly detection tools.

- Exposure to cloud platforms, especially AWS (S3, Redshift, Glue).

- Understanding of analytical data modelling concepts, including Star Schema and OLAP structures.

- Knowledge of testing AI/ML models, particularly for consistency, fairness, or accuracy in predictions.

Certifications Required : AWS Certified Solutions Architect, Certified Tester Foundation Level (CTFL)