Apply Now

Sr. Databricks Engineer (PySpark and Data Lake) Exp. || 5 Days Onsite

Wilmington, DE, US • Posted 3 hours ago • Updated 3 hours ago

Contract W2

Contract Corp To Corp

No Travel Required

On-site

Depends on Experience

Fitment

Dice Job Match Score™

👤 Reviewing your profile...

Job Details

Skills

Databricks

Summary

Job Title: Sr. Databricks Engineer (PySpark and Data Lake) Exp.

Location: Wilmington, DE – 5 Days onsite role

Duration: Full Time / Long Term Project

Overview

We are seeking a Data Engineer to lead the modernization of legacy ETL systems by migrating Ab Initio workflows to scalable, modular PySpark pipelines on Databricks. The role involves transforming complex data ecosystems into cloud-native architectures while ensuring data integrity, performance, and reliability.

Key Responsibilities

ETL Modernization & Development

Analyze and migrate legacy ETL workflows from Ab Initio to PySpark-based pipelines
Design and develop scalable data pipelines on Databricks
Refactor monolithic processes into modular, reusable components
Leverage existing enterprise datasets to avoid redundancy

Data Integration & Processing

Build and maintain ETL/ELT pipelines integrating data from Snowflake and other sources
Process and publish enriched datasets for downstream applications
Support batch and near real-time data processing

Data Lineage & Optimization

Create end-to-end data lineage and data flow diagrams
Identify redundancies and drive process consolidation and optimization
Ensure adherence to data governance and quality standards

Testing & Validation

Develop unit, integration, and reconciliation frameworks
Perform dual-run comparisons with legacy systems
Validate outputs in UAT and pre-production environments

Deployment & Operations

Support cutover and migration strategy from legacy systems
Decommission legacy workflows and optimize scheduling (e.g., Control-M)
Develop runbooks, monitoring, and operational documentation

Collaboration

Work with data architects, analysts, and downstream application teams
Coordinate user acceptance testing (UAT/FAT) and stakeholder sign-offs

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 10113809
Position Id: 106509-1090-146160
Posted 3 hours ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Wilmington, Delaware

•

Today

Job Title: Sr. Databricks Engineer (PySpark and Data Lake) Exp. Location: Wilmington, DE 5Days onsite role Duration: Full Time / Long Term Project Overview We are seeking a Data Engineer to lead the modernization of legacy ETL systems by migrating Ab Initio workflows to scalable, modular PySpark pipelines on Databricks. The role involves transforming complex data ecosystems into cloud-native architectures while ensuring data integrity, performance, and reliability. Key Responsibilitie

Easy Apply

Contract, Third Party

Depends on Experience

Data Scientist - Pharma Onsite

Wilmington, Delaware

•

Today

Hiring! Data Scientist with a strong Manufacturing background for our direct client located in Wilmington, Delaware. This is an onsite job requirement. Seeking Candidates that will work on a W2 basis for 12 month + contract. Collaborate with stakeholders to identify the business requirements and the expected outcome;Identify and select relevant and credible data sources, and provide technical instructions to data analysts to collect, integrate and mine data;Identify and analyze patterns. Design

Easy Apply

Contract

Depends on Experience

Python - Basic

Wilmington, Delaware

•

Today

Location: Wilmington, DE Description: Job Title: Python Developer Location: Wilmington DE Contract to hire role Responsibilities Design, build, and maintain Tableau dashboards and visualizations to support Risk and Control reporting and insights Identify manual or repetitive processes and develop Python-based automation solutions to improve efficiency and scalability Develop production-quality Python code to ingest, transform, validate, and analyze structured and semi-structured data Protot

Contract

Azure databricks

Wilmington, Delaware

•

26d ago

Key Responsibilities Design and implement ETL/ELT pipelines using Databricks and Apache Spark.Optimize data workflows for performance, scalability, and cost efficiency.Collaborate with data scientists, analysts, and business stakeholders to deliver data-driven solutions.Manage and monitor Databricks clusters, jobs, and workflows.Integrate Databricks with Azure/AWS/Google Cloud Platform services (depending on organization s cloud strategy).Ensure data quality, governance, and security compliance

Easy Apply

Full-time

Up to $120,000

Search all similar jobs

Sr. Databricks Engineer (PySpark and Data Lake) Exp. || 5 Days Onsite

Dice Job Match Score™

Job Details

Skills

Summary

Similar Jobs