Apply Now

Lead Databricks Engineer with Python (Only local to MN)

Eagan, MN, US • Posted 1 day ago • Updated 1 day ago

Contract W2

Contract Corp To Corp

Contract Independent

6 Months

No Travel Required

Able to Sponsor

On-site

Depends on Experience

Fitment

Dice Job Match Score™

📊 Calculating match score...

Job Details

Skills

Python
PySpark
Databricks
SQL
AWS

Summary

Lead Databricks Engineer (Contract-to-Hire)

Location: Eagan, MN (Hybrid – 2 days/week onsite)
Duration: 3–6 Months Contract-to-Hire
Start: ASAP

< data-start=168 data-end=184>Job Summary

We are seeking a Senior/Lead Data Engineer to modernize and enhance critical enterprise data pipelines by migrating legacy SAS workflows to Python/PySpark on Databricks. This role will initially work alongside SMEs during knowledge transfer and then assume ownership of a high-impact data platform supporting healthcare analytics and reporting.

< data-start=531 data-end=551>Required Skills

Strong Data Engineering and application development experience
Expert-level Python development and scripting
Hands-on experience with PySpark and Databricks
Experience building and maintaining large-scale distributed data pipelines
Strong SQL skills for complex data extraction, transformation, and optimization
Experience with Databricks notebooks, workflows, and cluster management
AWS experience with S3, Lambda, Glue, and EC2
Experience scheduling, automating, and monitoring data pipelines
Git/version control experience in collaborative development environments
Experience working with Agile tools, epics, and issue management

< data-start=1206 data-end=1223>Nice to Have

AI/ML, Generative AI, or Automation experience
Healthcare or HEDIS data experience
Lead or Principal-level Data Engineering experience

< data-start=1366 data-end=1391>Key Responsibilities

Migrate SAS-based pipelines to Python/PySpark on Databricks
Design, develop, and optimize scalable data processing solutions
Own and support critical enterprise data pipelines
Automate workflows and improve operational efficiency
Collaborate with business stakeholders, SMEs, and analytics teams
Ensure data quality, reliability, and performance across the platform

Must-Have Skills

Python Development
PySpark
Databricks
SQL
AWS (S3, Glue, Lambda, EC2)
Data Pipeline Development
Distributed Data Processing
Git/Version Control

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 10120317
Position Id: 9001125
Posted 1 day ago

Contact the job poster

Chandee Pandey

Recruiter! @ Sovereign Technologies

View Profile

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Senior Data Engineer

Remote or Minnetonka, Minnesota

•

Yesterday

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by diversity and inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us adv

Full-time

USD 91,700.00 - 163,700.00 per year

Databricks Engineer - Remote

Remote

•

6d ago

Title: Databricks Engineer - Remote Mandatory skills: operating solutions, Databricks, ETL, ELT architectures, data ingestion patterns, Python, Spark, data processing, dimensional data models, data lakehouse, AI-driven analytics platforms, semantic modeling, NLP techniques, data governance, data security, data-driven solutions, data technologies, data models data structures, data pipelines, data quality, SAFe Agile, reliable analytics solutions, SQL queries, notebooks, data ingestion, transforma

Easy Apply

Contract

55 - 60

Data Architect/ Enablement Lead (AWS, Databricks)

Remote

•

7d ago

Role: Data Enablement & Strategy Lead Remote and travel We are seeking a highly capable Data Enablement & Strategy Leadwho can combine strategic leadership with handson technical delivery. This role will champion data platform adoption, accelerate enterprise data initiatives, and guide teams in building scalable solutions on Databricks and AWS. The ideal candidate brings deep technical expertise, strong execution ownership, and the ability to influence and align crossfunctional stakeholders. K

Easy Apply

Full-time

Depends on Experience

Databricks Architect

Remote

•

Yesterday

CEI is seeking a Databricks Architect to join our growing organization! Client/Industry: CEI AI / IT Solutions/Consulting and Professional Services Job Title: Databricks Technical Architect Location: Hybrid in Pittsburgh, PA 15205 (3 Days On-Site / 2 Days Remote) | OR | Remote (1st pref-EST; 2nd pref-CST) Work Schedule/Shift: Monday to Friday, Standard business hours | Minimum expectation of ~40 work hours per week. Duration/Length of Assignment: W2 Direct Hire / Full Time Compensation: Competi

Easy Apply

Full-time

Depends on Experience

Search all similar jobs

More jobs at Sovereign Technologies in Eagan, MN