Sr Azure Data Engineer (Databricks)

Hi,

Greetings from DIA SOFTWARE SOLUTIONS LLC!

We reaching out about an exciting Direct client opportunity with one of our clients. Please review the requirements and let me know if you are interested in this position?

Direct client Req:: Need Sr Azure Data Engineer (Databricks) ,Remote

PLEASE SEND THE RESUMES TO SKUMAR AT DIASOFTWARESOLUTIONS DOT COM !

Job Description:

Need 12+ Only

Data Engineering, SQL, Python, ADF, Databricks, Azure

Data Engineering, Databricks, SQL, ETL, ADF, Lead experience, Ownership experience

Note: Looking for candidates with strong hands-on experience in required tech stack along with being a person to take ownership, lead expertise, planning and designing data pipelines, data related infra for new product, can operate at more strategical level.

Roles and Responsibilities

Design, build, and support end-to-end data pipelines (ingestion, transformation, validation, publishing). Develop and optimize SQL and PySpark/Databricks transformations for large datasets.

Build production-grade Python components (reusable modules, logging, error handling, testing). Create and maintain Azure Data Factory (ADF) pipelines (triggers, parameterization, monitoring, failure handling).

Work within Azure environments (ADLS Gen2, Azure SQL, resource groups, portal operations).

Provision and maintain Azure components using Pulumi (Infrastructure as Code). Participate in code reviews, documentation, and operational support (triage + root cause analysis)

Mandatory skills

ETL / ELT Concepts: Strong understanding of pipeline patterns, incremental loads, data validation, and troubleshooting.

SQL: Advanced querying (CTEs, views, joins, complex query logic) and performance tuning for transformations and validation. Python: Production-quality development (modular code, testing, logging, integration with APIs/files, CICD, Unit Test/Integration test automation, Code Coverage).

PySpark: Distributed transformations and performance optimization (joins, partitions, debugging), CICD, Unit Test/Integration test automation, Code Coverage.

Azure Data Factory (ADF): Build/operate ADF pipelines, parameterization, triggers, monitoring, retry/error handling; integrate with Databricks/ADLS.

Databricks: Develop and operationalize notebooks/jobs/workflows; Delta Lake patterns; basic cluster/job configuration.

Azure Fundamentals + Pulumi: Hands-on with ADLS Gen2, Azure Portal, Storage Explorer, Resource Groups, Azure SQL, and familiarity integrating with Azure OpenAI. Able to use/maintain Pulumi scripts for provisioning and managing Azure resources across environments

DIA SOFTWARE SOLUTIONS LLC.

SANGEETHA| DIA SOFTWARE SOLUTIONS LLC.

Direct: 

DIA SOFTWARE SOLUTIONS is an Affirmative Action/Equal Opportunity Employer that supports workplace diversity. All employment decisions are made without regard to race, color, religion, sex, national origin, age, disability, veteran status, marital or family status, sexual orientation, gender identity, or genetic information. All Dia soft staff must be able to demonstrate the legal right to work in the United States. DIA SOFTWARE SOLUTIONS is an E-Verify employer