QA Engineer (SDET - Databricks and reporting tool automation)

Overview

Accepts corp to corp applications
Contract - Independent
Contract - W2
Contract - Month C2H

Skills

PowerBI
SDET
Databricks
QA Engineer

Job Details

We are seeking a Senior QA Engineer / SDET with strong experience in data lake validation, Databricks automation, and reporting tool testing. The ideal candidate will be responsible for building and executing test strategies that ensure data quality, reliability, and compliance across complex data pipelines at enterprise scale.

This role will focus heavily on data validation, functional testing of Databricks transformations, and automation of data quality checks. The candidate must have prior hands-on experience with data lake validations, Spark/Python scripting, and Azure cloud tools.

Requirements:

  • Proven hands-on experience with Databricks test automation, Spark, Python, and Azure.
  • Strong background in data validation for data lakes and transformations.
  • Experience with automation frameworks, functional testing, and CI/CD integration.
  • Knowledge of Kusto (ADX), SQL, and Parquet/Delta data formats.
  • Some UI automation experience is a plus (ability to learn quickly).
  • Familiarity with financial transaction/payment validations strongly preferred.
  • Excellent communication skills: ability to operate at a senior level, explain strategy, and demonstrate prior impact.
  • Senior level QA/SDET with a proven track record of delivering automation in Databricks and Azure ecosystems
  • Quick to learn functional and technical aspects of new systems
  • Strong communicator

Key Responsibilities

Own the test strategy, implementation, and automation for batch and streaming data pipelines on Databricks, Azure Data Explorer (ADX), and cloud data lakes.

Build automated tests and quality gates for data ingestion, transformation, and output validation (ADX/Delta/Parquet/SQL).

Integrate test automation into CI/CD pipelines.

Partner with data engineers, platform, and analytics teams to ensure correctness, reliability, and compliance.

Design and automate functional validation of Databricks transformations (modern equivalent of ETL validation).

Create and execute use cases for payments and transactions (CC authorization, authorization, purchases, modifications, domain-specific data).

Perform validation for data logs, reporting outputs, and PowerBI dashboards.

Support legacy test scripts when necessary, while driving modernization.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.