ETL /BI TESTER

Overview

Hybrid
65 - 75
Contract - W2
Contract - 12 Month(s)
No Travel Required
Unable to Provide Sponsorship

Skills

DATABRICKS
Data Governance
Data Engineering
ELT
Quality Assurance
Python

Job Details

We are seeking an experienced QA and Data Governance Lead to oversee quality assurance and data governance practices across our Data & Analytics organization. This role is hands-on person leading QA engineers, defining and enforcing QA best practices, implementing automated testing frameworks (Databricks, Python), and driving data governance initiatives to ensure data quality, compliance, and trustworthiness across the enterprise.

Job Description:

·         Define and implement QA strategy, standards, and best practices for data engineering, analytics, and reporting pipelines.

·         Lead and mentor a team of QA engineers, providing guidance on automation frameworks, testing methodologies, and continuous improvement.

·         Develop, maintain, and optimize automated test scripts using Databricks, Python, and CI/CD frameworks for data validation, regression, and performance testing.

·         Collaborate with data engineering teams to validate ETL/ELT pipelines, medallion architecture layers (bronze/silver/gold), and data transformations.

·         Establish monitoring and alerting data quality issues across pipelines.

·         Drive adoption of test-driven development (TDD) and automated validation in the data lifecycle.

·         Lead the development and implementation of data governance frameworks, policies, and standards across the organization.

·         Partner with data stewards, business teams, and compliance to define data ownership, metadata management, lineage tracking, and access controls.

·         Implement data quality metrics, KPIs, and scorecards to measure and improve trust in enterprise data assets.

·         Ensure compliance with regulatory requirements (GDPR, CCPA, SOX, etc.) and company data governance policies.

·         Familiarity with data structures, storage systems, cloud infrastructure, and other technical tools.

·         Ability to work effectively in teams of technical and non-technical individuals.

·         Ability to continuously learn, work independently, and make decisions with minimal supervision

·         Demonstrate accountability, prioritize tasks, and consistently meet deadlines.

·         Familiarity with Agile/SCRUM practices

·         Work with cloud platforms such as AWS, Azure, or Google Cloud to integrate and manage data workflows and storage solutions.

Scope of work:

·         Strong expertise in Databricks, including Spark (PySpark/Scala).

·         Hands-on experience working with cloud platforms (AWS, Azure, or Google Cloud).

·         Experience with CI/CD tools like Azure DevOps, GitHub Actions, or Terraform for infrastructure-as-code.

·         Familiarity with data modeling, data warehousing, and database design.

·         Strong understanding of data formats such as Parquet, ORC, JSON, and Avro.

·         Ability to work in an agile environment and adapt to changing requirements.

·         Strong analytical and problem-solving skills with a focus on optimizing data engineering solutions.

·         Study, analyze and understand business requirements in context to business intelligence.

·         Design and map data models to shift raw data into meaningful insights.

·         Spot key performance indicators with apt objectives

·         Make essential technical and strategic changes to improvise present business intelligence systems

·         Identify the requirements and develop custom charts accordingly

·         SQL querying for better results

Credentials:

Bachelor’s degree in computer science, Information Systems, Data Management, or related field.

8+ years of experience in QA, data testing, or data engineering, with 3+ years in a leadership role.

·         Hands-on expertise in Databricks, Python, SQL, and data pipeline testing automation.

·         Proven experience in leading QA teams and establishing QA standards in a data/analytics environment.

·         Strong knowledge of data governance frameworks and best practices.

·         Familiarity with data catalog, metadata, and lineage tools.

·         Strong understanding of ETL/ELT testing, medallion/lakehouse architecture, and cloud-based data platforms (Azure, AWS, Google Cloud Platform).

·         Excellent communication, leadership, and stakeholder management skills.

 

Required:

·         Minimum 10 years of experience of working on Data platforms.

·         Knowledge about database management, SQL querying, data modeling, and Online Analytical Processing.

·         Additional consideration given for experience scripting and programming language such as Python

·         Leadership & Team Development

·         Strong Analytical & Problem-Solving Skills

·         Communication & Stakeholder Management

·         Strategic Thinking with Hands-On Technical Execution

·         Data Quality & Governance Advocacy

·         Ability to mentor junior team members and contribute to technical leadership

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.