Senior Azure Data Engineer

Remote • Posted 1 hour ago • Updated 1 hour ago
Contract W2
12 Months
Remote
Depends on Experience
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • ADF
  • databricks
  • PySpark

Summary

Senior Azure Data Engineer

Location: Atlanta, GA (Open for Remote)

Type: [Full-time / Contract] | Start: Immediate

We're looking for a hands-on Senior Azure Data Engineer to build and run production data pipelines on Azure. This role focuses on building complex data pipelines using Azure Data Factory and Azure Databricks, along with data validation using Python, PySpark, Pandas, and Great Expectations.

What You'll Do:

  • Analyze incoming data files (CSV, JSON, Parquet) and their schemas to plan ingestion and transformation.
  • Design and build lakehouse-style Medallion (Bronze, Silver, Gold) data pipelines, using Azure Data Factory for data movement and Azure Databricks for complex data processing.
  • Build and maintain complex, multi-step pipelines in Azure Data Factory.
  • Use ADF to trigger and manage Azure Databricks Jobs, including passing parameters, tracking job status, and handling failures and retries.
  • Write data transformation and validation logic in Python and PySpark.Set up automated data quality checks using Great Expectations and Pandas, and flag bad data before it reaches downstream users.
  • Troubleshoot and tune pipelines in production.

Core Skills Required-

  1. 7+ years in data engineering, with recent senior-level work.
  2. Strong hands-on experience with Azure Data Factory and its integration with Azure Databricks Jobs.Strong Python, PySpark, and SQL skills.
  3. Hands-on experience building data validation frameworks with
  4. Great Expectations and Pandas.
  5. Solid cloud data engineering background. Azure preferred; AWS or Google Cloud Platform considered.

Big Plus-

  1. Local to Atlanta, GA / Open for remote too
  2. Experience on State or Federal government projects.
  3. Azure Data Engineer (DP-203) or Databricks certification.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 91092411
  • Position Id: 9107899
  • Posted 1 hour ago
Contact the job poster
RS

Rachel Singh

Recruiter @ kr3 information systems inc
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Remote or Almont, Colorado

•

Today

Contract

Remote or Almont, Colorado

•

Today

Contract

Remote

•

8d ago

Easy Apply

Contract

$80 - $90

Remote

•

Today

Full-time

Search all similar jobs