Senior ETL Engineer Databricks & Spark SQL Expert - SE 25-28506

Overview

Remote
Depends on Experience
Accepts corp to corp applications
Contract - W2
Contract - Independent
No Travel Required

Skills

XML
Python
SQL
Scalability
Software Engineering
Version Control
Workflow
IBM Lotus Domino
JSON
Java
Management
Orchestration
Problem Solving
Data Science
Data Storage
Databricks
Debugging
Extract
Transform
Load
Conflict Resolution
Continuous Delivery
Continuous Integration
Data Engineering
Amazon S3
Amazon Web Services
Apache Spark
Cloud Computing
Collaboration
Git

Job Details

Job Title: Senior ETL Engineer Databricks & Spark SQL Expert
Location: Remote
Employment Type: Contract
Experience Level: Mid to Senior (8+ years)

About the Role

We are seeking a highly motivated and experienced ETL Engineer with deep expertise in Databricks, Apache Spark, and cloud-based data engineering. The ideal candidate will have extensive hands-on experience designing and scaling ETL pipelines, advanced proficiency in Spark SQL, and strong Python programming skills. This role also requires proven experience with AWS services and Domino Data Lab for collaboration in data science and engineering workflows.

Key Responsibilities

  • Design, build, and maintain scalable ETL pipelines using Databricks and Spark SQL.

  • Process large-scale structured and semi-structured data, including complex JSON and XML.

  • Optimize Spark jobs for performance and scalability within Databricks.

  • Troubleshoot and improve reliability using in-depth knowledge of Databricks internals.

  • Integrate and manage data using AWS services: S3, IAM, Secrets Manager, Glue, ECS/EKS.

  • Develop clean, efficient, and testable Python code for data workflows.

  • Collaborate with data scientists, analysts, and engineers to meet business objectives.

  • Work within Domino Data Lab for model development and team collaboration.

Required Qualifications

  • 4+ years of experience in Databricks, with advanced knowledge of platform internals and configuration.

  • 5+ years of experience with Apache Spark, including expert-level Spark SQL skills.

  • Proficiency in Python for ETL and pipeline development.

  • Strong hands-on experience handling complex JSON and XML structures.

  • 3+ years of experience with Domino Data Lab.

  • Solid working knowledge of AWS services, including:

    • S3 (data storage and access)

    • IAM (security and permissions)

    • Secrets Manager (credential management)

    • AWS Glue (ETL orchestration)

    • ECS/EKS (containerized workload deployment)

  • Strong troubleshooting and debugging skills for large-scale data systems.

  • Self-starter with excellent problem-solving skills and ability to work independently.

Preferred Qualifications

  • 10+ years of Java development experience (strong plus).

  • Familiarity with CI/CD pipelines and version control tools (e.g., Git).

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.