Data Engineer with Databricks and PySpark

Minneapolis, MN, US • Posted 19 hours ago • Updated 12 hours ago

Full Time

On-site

Fitment

Dice Job Match Score™

🎯 Assessing qualifications...

Job Details

Skills

Collaboration
Data Processing
Scala
Extract
Transform
Load
Unstructured Data
Cloud Computing
Management
Databricks
Data Quality
Oracle
Migration
Informatica
PySpark
Database
Unix

Summary

Data Engineer with Databricks and PySpark

Minneapolis, MN

Onsite from day 1

Design, develop, and optimize data pipelines and transformations using PySpark and Databricks.
Collaborate with data architects and analysts to define and implement data processing needs.
Write production-grade PySpark/Scala code for ETL and data transformation.
Integrate structured and unstructured data across cloud platforms.
Manage and configure the Databricks environment, including clusters and notebooks.
Troubleshoot and resolve issues related to Databricks performance and functionality.
Exp in DLT Framework is critical
Exp in Expectation framework is essential (Implement data quality checks and monitor processes.)
Knowledge in Migration Project
Informatica and Oracle skill is mandatory as the Program is migration from Informatica to Pyspark in DB
Good exp in AIRFLOW DAG creation, UNIX

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 10236892
Position Id: OOJ - 4468-3469-1774277627
Posted 19 hours ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Senior Data Engineer-MN/WI Local Hybrid

Hybrid in Minneapolis, Minnesota

•

Today

Title - Senior Data Engineer Location-MN/WI Local Hybrid Duration Contract Overview Looking for senior, hands-on data engineers to support a modern data platform initiative. This is a code-first engineering role not traditional ETL or reporting. Candidates with experience in health payer environments are strongly preferred specifically supporting data across Clinical, Claims, or Provider domains. Ideal candidates understand how to work with healthcare data models, business logic, and large-sca

Easy Apply

Contract

Depends on Experience

Senior Data Engineer MN/WI Local Hybrid In person Interview

Hybrid in Minnetonka, Minnesota

•

3d ago

Senior Data Engineer MN/WI Local Hybrid In person Interview Long Term Contract Looking for senior, hands-on data engineers to support a modern data platform initiative. This is a code-first engineering role not traditional ETL or reporting. Candidates with experience in health payer environments are strongly preferred specifically supporting data across Clinical, Claims, or Provider domains. Ideal candidates understand how to work with healthcare data models, business logic, and large-scale dat

Easy Apply

Contract, Third Party

Depends on Experience

Lead Data Engineer - Hybrid and Contract to Hire Opportunity

Eagan, Minnesota

•

Today

Genesis10 is currently seeking a Lead Data Engineer for a 9-month contract-to-hire role with our healthcare client in Eagan, MN. This role will be hybrid, working 2 days onsite. Compensation: $85.00 - 95.00 per hour, depending on skill and experience level. We are hiring a Lead Data Engineer for Client of MN! This position is responsible for providing technical and resource leadership to enable data analytics with structured and unstructured data. You will be responsible for leading or executing

Easy Apply

Full-time

Compensation information provided in the description

Solutions Architect - (Data Engineering)

Minneapolis, Minnesota

•

Today

Shape the Future of Data & AI - Architect Solutions That Scale This Jobot Job is hosted by: Robert Donohue Are you a fit? Easy Apply now by clicking the "Apply Now" button and sending us your resume. Salary: $140,000 - $170,000 per year A bit about us: We're a global leader in data and AI/ML services and one of the fastest-growing companies in the modern data stack space. We partner with top cloud platforms like Snowflake, AWS, Azure, Google Cloud Platform, Databricks, Fivetran, Pinecone, Gle

Full-time

USD 140,000.00 - 170,000.00 per year

Search all similar jobs