Senior Data Engineer - NO C2C/1099

Temp to Hire (no visa or 1099 candidates)

Client: EdTech

Title: Senior Data Engineer – Data Analytics and AI Focused

Duration: 6 Months (Contract to Hire)

Location: 100% Remote

Key Responsibilities

·        Define the end-to-end testing scope based on solution architecture and project documentation

• Design and implement a comprehensive testing strategy and plan aligned with organizational QA standards
• Develop and maintain test scripts and frameworks for the Redshift serverless platform
• Perform testing across key technologies, including:

o   AWS Redshift

o   AWS DMS (Data Migration Service)

o   AWS Glue

o   PySpark Deequ

o   Event Bridge

o   Data Lakes

o   Python-based data pipelines

o   Apache Airflow

o   dbt (data build tool)

• Build and implement automated testing solutions to ensure:

o   End-to-end data validation

o   Data ingestion accuracy

o   Transformation logic integrity

o   Data pipeline reliability

• Conduct test coverage analysis and ensure adequate validation across all data engineering workflows
• Prepare and manage test data
• Review and provide feedback on:

o   Solution architecture

o   Data models

o   Design and technical documentation

• Collaborate with cross-functional teams (Data Engineering, BI, DevOps, Product) to:

o   Identify testing impacts

o   Mitigate risks

o   Ensure high-quality deliverables

Required Qualifications

·        Proven experience in data engineering testing / data QA / ETL validation

·        Strong hands-on experience with AWS data services (Redshift, Glue, DMS)

·        Proficiency in Python for test automation and validation

·        Experience with Airflow and orchestration testing

·        Hands-on experience with dbt and data transformation validation

·        Familiarity with CDK for infrastructure validation

·        Experience in BI testing in Quicksuite will be highly beneficial

·        Experience with data quality tools such as PySpark Deequ or similar

• Strong understanding of:

o   Data warehousing concepts

o   ETL/ELT pipelines

o   Data validation techniques (schema, reconciliation, anomaly detection)

Preferred Qualifications

·        Experience designing enterprise-level test strategies for data platforms

·        Knowledge of CI/CD pipelines for data and test automation

·        Experience working in Agile / Scrum environments

·        Familiarity with data observability frameworks