Azure Databricks Engineer 10+

Pleasanton, CA, US • Posted 2 days ago • Updated 2 days ago

Contract W2

No Travel Required

Able to Sponsor

On-site

$48 - $48/hr

Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

SQl
Azure Databricks
Apache Spark

Summary

Job Description: Senior Azure Databricks Engineer

Location Pleasanton, CA (Onsite)

EXP: 10 Years

About the Role

We are seeking a highly experienced Senior Azure Databricks Engineer to support digital transformation initiatives for our client. The ideal candidate has strong expertise in Azure cloud services, Databricks, Spark, large-scale data engineering, and end-to-end pipeline development. This is an onsite role requiring candidates to work from any of the client locations listed above.

Responsibilities

Design, build, and optimize end-to-end data pipelines using Azure Databricks and Apache Spark.
Develop scalable ETL/ELT processes for structured and unstructured data.
Integrate Databricks with various Azure services (ADLS, ADF, Azure SQL, Synapse, Key Vault, etc.).
Implement Delta Lake features—time travel, ACID transactions, schema evolution, and optimization.
Perform advanced data transformation, cleansing, aggregation, and feature engineering.
Optimize Spark jobs for performance, reliability, and cost efficiency.
Analyze and troubleshoot complex data issues in production environments.
Collaborate with data architects, analysts, and business teams to define requirements & deliver solutions.
Follow CI/CD best practices for data engineering using Git, Azure DevOps, or similar tools.
Document solutions, create technical specifications, and support code reviews.

Required Skills

10years of professional Data Engineering experience.
Strong hands-on expertise with Azure Databricks and Apache Spark (PySpark / Scala).
Strong experience with Azure cloud services:
Azure Data Lake (ADLS)
Azure Data Factory (ADF)
Azure SQL / Synapse
Azure Key Vault
Strong SQL development skills (complex queries, window functions, tuning).
Experience with Delta Lake features and Databricks workspace management.
Experience building and running pipelines on CI/CD frameworks (Azure DevOps preferred).
Strong understanding of data modeling, partitioning, performance tuning, and job orchestration.
Experience working with large datasets in distributed environments.
Excellent communication and problem-solving skills.

Education: Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91159647
Position Id: 8921858
Posted 2 days ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Data Engineer with Databricks

Santa Clara, California

•

3d ago

Role: Data Engineer with Databricks/Spark and Gen AI/LLM Experience Location: Santa Clara, CA (Onsite from Day 1) Experience: 12-14 year max About the Role We are seeking a skilled Data Engineer with hands-on experience in Databricks to join our team in Santa Clara. The ideal candidate will have a strong background in data engineering, excellent coding skills, and the ability to work collaboratively in a fast-paced environment. Responsibilities: - Design, develop, and maintain scalable data pi

Easy Apply

Contract, Third Party

Depends on Experience

Data Architect (Lakehouse)

San Jose, California

•

14d ago

Job Title: Data Architect (Lakehouse) San Jose California 12+ Months About the Role : We are seeking an experienced Data Architect to serve as the technical authority on our data platform team. Lead Data Engineer/Architect will be responsible for shaping the architecture and engineering standards of our enterprise-scale Lakehouse environment on Databricks, guiding a team of talented engineers, and driving measurable improvements in platform performance and cost efficiency. Candidate Profile 8-10

Easy Apply

Contract, Third Party

Depends on Experience

Databricks Data Engineer

San Francisco, California

•

16d ago

Role: Databricks Data Engineer Location: San Francisco/ Seattle ( 5 Days Onsite) JD We are seeking multiple positions (10 years of experience) in Databricks along with Adv SQL Knowledge The primary responsibility of this role is to provide Technical support for the Databricks Environment. Must Have Atleast 10+Years of Experience Local to CA Key Responsibilities: Bug Fixes in the Databricks environment Ability to Monitor, Transform and optimize ETL pipelines for Databricks and Knowledge of Data

Easy Apply

Full-time

Depends on Experience

DataBricks Tech Lead

Remote

•

3d ago

Tech Lead - DataBricks As a Tech Lead, you will work closely with business stakeholders, solution architects, product owner, scrum master, and subject matter experts (SMEs) to understand requirements and deliver high-quality, scalable solutions on Azure Databricks. Responsibilities Summary: Lead solution design and delivery for Databricks-based platforms and products (batch/streaming, BI-ready models, ML workflows). Translate business requirements into reference architectures, implementation pla

Easy Apply

Contract

Depends on Experience

Search all similar jobs