Apply Now

Databricks Automation Engineer / ETL Tester

Addison, TX, US • Posted 12 days ago • Updated 2 hours ago

Contract W2

On-site

Depends on Experience

Thunderhawk Technology Partners

Fitment

Dice Job Match Score™

📋 Comparing job requirements...

Job Details

Skills

Summary

Role: Databricks Automation Engineer
Location: Dallas, TX

Roles and Responsibilities:

Define and own cross-team test architecture and frameworks for ETL/ELT, Databricks/Spark, APIs, and UIs that scale across multiple product teams and environments.
Design and implement comprehensive data validation and data-quality strategies for large-scale telemetry processing (schema validation, row counts, statistical checks, delta comparisons, anomaly detection, drift detection).
Architect and enforce CI/CD-integrated automated testing pipelines (Azure DevOps / GitHub Actions) supporting pre-deploy gating, contract testing, canary verification, and regression protection.
Lead synthetic test data generation strategies and robust mocking/stubbing approaches for
downstream dependencies; ensure environment-aware and secure test harnesses for sensitive telemetry.
Drive performance, scalability, and distributed load testing for Spark jobs and supporting microservices; quantify bottlenecks and propose system- and code-level remediation.
Collaborate with data engineering, platform, DevOps, and product leadership to define acceptance criteria, release quality gates, and automated verification SLAs.
Implement and maintain observability for test executions; publish automation health metrics (passrates, flakiness, coverage, mean time to detection) and maintain executive-facing dashboards.
Own test documentation, runbooks, failure triage playbooks, and on-call handover for production incidents related to data correctness and test infrastructure.
Lead and grow a high-performing QA/test automation capability — hire, coach, establish standards, and promote best practices organization-wide

Must Have Skills:

8+ years of professional software engineering and testing experience, including technical ownership of QA/test practices across the SDLC.
3+ years leading or mentoring engineering/QA teams in technical areas (Test Lead, Principal QA Engineer, or similar).
Deep knowledge of data engineering and lakehouse patterns: partitioning strategies, ACID/Delta semantics, schema evolution, delta table optimization, CDC, time-travel, and data lineage; able to design verification approaches for Databricks/Spark/Delta Lake systems.
Proven experience architecting and building scalable automated test frameworks and embedding them into enterprise CI/CD using Azure DevOps or GitHub Actions.
Strong software engineering skills with extensive coding experience in Python, Java, or Scala for test development and automation; ability to review and contribute production-grade code.
Extensive experience testing data pipelines and big-data processing (Databricks, Spark/PySpark, Delta Lake, ADLS Gen2) or equivalent streaming/batch platforms at scale.

Thanks
Thunderhawk Technology Partners LLC

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91170649
Position Id: 26-03205
Posted 12 days ago

Company Info

About Thunderhawk Technology Partners

Thunderhawk Technology Partners is a woman and minority-owned business, headquartered in Atlanta, GA, committed to reshaping the future of our workforce. Our story is one of passion and purpose, built on decades of experience in workforce management and a shared belief in empowering talent. We are dedicated to delivering innovative solutions that drive client success while fostering an inclusive culture where our employees thrive.

With both global reach and local expertise, we offer a range of services across Technology, Engineering, Business Operations, and Healthcare. Our commitment to diversity, equity, and inclusion is at the heart of what we do, reflected in programs like Warrior to Workforce, Returnship (EmpowerHer, Pathway to Success, Experience Matters) and Neuro-Inclusion.

At Thunderhawk, we treat people with care and respect, striving to build high-performing teams that reflect the values of our clients and the communities we serve. Let us be your trusted partner in shaping a more inclusive and successful future for your organization.

Go to company profile

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

QA Analyst with Power BI

Jacksonville, Florida

•

13d ago

Job Title: QA Analyst Power BI Location: Jersey City, NJ (Onsite/Hybrid) Experience: 810 Years Job Description: We are looking for an experienced QA Analyst with strong expertise in Power BI reporting, validation, and enterprise analytics across Healthcare, P&C Insurance, and Banking domains. Key Responsibilities:Perform QA testing and validation of Power BI dashboards and reportsValidate data accuracy using SQL queriesConduct KPI reconciliation and reporting validationTranslate business requ

Easy Apply

Contract

$45.45

Search all similar jobs