Lead Azure Databricks Engineer @ Dallas, TX--Onsite

Dallas, TX, US • Posted 2 hours ago • Updated 2 hours ago
Contract W2
12 Months
No Travel Required
On-site
Depends on Experience
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Lead Azure Databricks Engineer

Summary

Hello,

I am Mohammed Dastagir with Saxon Global Inc wanted to let you know about the job opportunity for Lead Azure Databricks Engineer position if interested please share your updated resume along with expecting rate.

 

Job Title: Lead Azure Databricks Engineer
Location: Dallas, TX--Onsite
duration: 12 + months contract

Only on W2

 

Job Summary

We are looking for an experienced Azure Databricks Data Engineer with strong expertise in designing, developing, and implementing scalable data engineering solutions using Microsoft Azure and Databricks. The ideal candidate should have hands-on experience in building ETL/ELT pipelines, processing large datasets, implementing data lakehouse architectures, and optimizing data workflows.

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Azure Databricks.
  • Develop data transformation workflows using PySpark, Spark SQL, and Python.
  • Build and manage ETL/ELT pipelines to process structured and unstructured data.
  • Work with Azure Data Factory (ADF) for data orchestration and pipeline scheduling.
  • Implement data lake and lakehouse solutions using Azure Data Lake Storage Gen2.
  • Work with Delta Lake, Delta Tables, and Medallion Architecture (Bronze, Silver, Gold).
  • Perform data cleansing, transformation, aggregation, and validation.
  • Optimize Spark jobs for performance, scalability, and cost efficiency.
  • Implement data security, governance, and access controls using Unity Catalog.
  • Integrate data from multiple sources, including SQL databases, APIs, files, and cloud storage.
  • Monitor, troubleshoot, and resolve data pipeline failures.
  • Collaborate with data architects, analysts, and business stakeholders.
  • Implement CI/CD processes using Azure DevOps and Git.

Required Technical Skills

  • Strong experience with Azure Databricks.
  • Hands-on experience with PySpark and Apache Spark.
  • Strong programming skills in Python and SQL.
  • Experience with Azure Data Factory (ADF).
  • Good knowledge of Azure Data Lake Storage Gen2 (ADLS).
  • Experience with Delta Lake and Delta Live Tables / Lakeflow Declarative Pipelines.
  • Understanding of data warehousing and data lakehouse concepts.
  • Experience in building batch and incremental data processing pipelines.
  • Knowledge of Spark optimization techniques, partitioning, and caching.
  • Experience with Azure DevOps, Git, and CI/CD pipelines.

Preferred Skills

  • Experience with Databricks Unity Catalog and data governance.
  • Knowledge of Databricks Workflows and job scheduling.
  • Experience with Azure Synapse Analytics.
  • Knowledge of dimensional modeling and data warehousing.
  • Experience with streaming technologies such as Structured Streaming and Event Hubs.
  • Familiarity with Terraform and Infrastructure as Code.
  • Understanding of cloud security, RBAC, and managed identities.
  • Experience with Power BI and downstream data consumption.

Qualifications

  • Bachelor's degree in Computer Science, Information Technology, or a related field.
  • 10 years of experience in Data Engineering.
  • Strong analytical and problem-solving skills.
  • Excellent communication and collaboration skills.

Key Competencies

  • Azure Cloud Data Engineering
  • Azure Databricks Development
  • PySpark and Spark SQL
  • ETL/ELT Development
  • Delta Lake and Lakehouse Architecture
  • Data Pipeline Optimization
  • Data Integration and Transformation
  • Data Governance and Security
  • CI/CD and DevOps

 

 

Mohammed Dastagir

Sr Resource Manager

Saxon Global Inc.

a:

Linked in:

w:   e: dastagir.m

 

 

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10167987
  • Position Id: 143946-703-1795206697
  • Posted 2 hours ago
Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.