Azure Data lead - Python, Pyspark, Databricks, ADF, Data Lake

New York City, NY, US • Posted 1 hour ago • Updated 1 hour ago
Contract W2
Contract Corp To Corp
On-site
Company Branding Image
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Python
  • Business Requirements
  • Translate
  • Posting
  • Governance
  • Data Manipulation
  • Data Quality
  • SQL
  • Database Modeling
  • SQL Queries
  • ETL
  • Best Practices
  • Problem-Solving
  • Oracle
  • Mentoring
  • Large-Scale
  • OOP
  • Design Patterns
  • Scripting
  • Object-Oriented
  • B2B Software
  • Data Modeling
  • Data Pipelines
  • PL/SQL
  • Performance Optimization
  • Caching
  • Apache Spark
  • Data Integration
  • Reconciliation
  • Data Warehousing
  • Datasets
  • WEB Database

Summary

Azure Senior Data Lead with Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, Python

Location: New York, NY

Experience: 7-12 Years

Primary Skills: Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, Python

Certifications: Relevant certifications in Azure Data Factory, Azure Databricks, SQL, Oracle, or Python would be a plus.

Other Requirement:

Azure Data Lead - Python, PySpark, Databricks, ADF, Data Lake

Job Description

Azure Senior Data Lead leads modernization of Python applications into scalable PySpark solutions on Azure Databricks.

Role Purpose

Lead the modernization and migration of existing Python object-oriented applications into scalable PySpark and Spark SQL data-processing solutions on Azure Databricks - bringing a strong blend of software engineering, data engineering, cloud architecture, and performance optimization.

Key Responsibilities
  • Analyze existing Python OOP applications and redesign single-node processing logic for distributed Spark execution.
  • Design, develop, and deploy enterprise-scale data pipelines on Azure Databricks; build reusable PySpark frameworks and utility modules.
  • Implement Delta Lake solutions using the Bronze Silver Gold architecture.
  • Build robust ETL/ELT pipelines with Azure Data Factory, ADLS Gen2, and Azure Synapse Analytics.
  • Implement data quality, reconciliation, validation, and monitoring frameworks.
  • Optimize Spark jobs using partitioning, bucketing, caching, broadcast joins, Adaptive Query Execution, and Delta optimization.
  • Benchmark converted applications against original Python implementations.
Core Skills
  • Python (expert), OOP, and advanced Python design patterns
  • PySpark, Spark SQL, and SQL
  • Azure Databricks, Azure Data Factory, ADLS Gen2
  • Apache Spark, Delta Lake, Data Lakehouse architecture, distributed computing
Must-have Skills
  • Python
  • Azure Databricks
  • Azure Data Factory (ADF)
  • MS SQL
  • Oracle PL/SQL
Good to Have
  • PySpark
  • Certifications in Azure Data Factory, Azure Databricks, SQL, Oracle, or Python
Experience & Expected Outcome

Senior data engineering leader with proven delivery of large-scale Databricks modernization programs.

Expected outcome: Existing Python applications converted into scalable, cost-efficient, enterprise-grade data solutions on Azure Databricks with proven performance parity.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10515154
  • Position Id: 2026-7161
  • Posted 1 hour ago

Company Info

About Aroha Technologies

Michael Arrington, founder and co-editor of Tech Crunch once famously said that ‘The best startups generally come from somebody needing to scratch an itch’.

 Quite true to the thought, Aroha Technologies Inc was launched in 2008 in Florida. While we began our journey with a powerful Staffing Specialist & Software Development, Today our procedure has amplified into a more robust and simplified solution for Marketers looking for consistent and long term.

In our quest to do better, we resolve to simplify user acquisition, retention, engagement and large Business without blinking an eyelid! A semi-remote team spread across the globe, we are here to help you one step at a time.

Unmatchable growing excellence When AROHA was formed, we interviewed many potential clients and business colleagues and asked what a single important question seemed: “If you could design a firm to meet your needs, what would it look like?” The answers to that question have guided our thinking and planning on an ongoing basis, and form the core of our thinking about how we treat our clients and our employees, whom we call associates.

At a most basic level, we have stated our mission in a promise to our clients and our associates: “Unmatchable growing excellence.” This alignment governs superior service to our customers and supports our associates who perform that work, which cannot be matched by our competitors.

In short, we believe that care of our associates, disciplined financial management, and constant attention to the fundamentals of good delivery adds up to a successful, scalable business.

About_Company_OneAbout_Company_Two
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

New York, New York

Today

Easy Apply

Contract, Third Party

Holmdel, New Jersey

Today

Easy Apply

Contract, Third Party

Search all similar jobs