Azure Data Engineer

Role: Azure Data Engineer

Location: Jackson, MI
Work Arrangement: Hybrid 3 days onsite per week

Job Description

We are looking for an experienced with strong hands-on expertise in Azure Synapse Analytics, Azure Data Factory (ADF), Databricks, SQL Server, Python/PySpark, and Azure cloud services. The ideal candidate will have experience designing and supporting enterprise data pipelines, developing complex SQL solutions, processing large datasets using Spark/Databricks, and deploying solutions through CI/CD pipelines.

Key Responsibilities

Design, develop, and maintain data solutions using Azure Synapse Analytics, Azure Data Factory, Databricks, Azure Storage, SQL Pools, and Azure Key Vault.
Develop ADF/Synapse linked services, datasets, pipelines, activities, and integration runtimes.
Build complex data workflows using ForEach, If, Switch, Until, Lookup, Stored Procedure, Script, Validation, Copy Data, and Data Flow activities.
Develop and maintain SQL Server stored procedures, functions, dynamic SQL, and complex data-processing logic.
Work with Spark Pools, Spark, PySpark, and Databricks notebooks for large-scale data processing.
Develop incremental/delta-load and data-merging solutions through notebooks.
Develop Python solutions using libraries such as PySpark, Pandas, NumPy, PyMySQL, and other data-processing libraries.
Retrieve and process data through REST APIs and stored procedures using Python.
Debug Python/PySpark code and troubleshoot library, dependency, and runtime issues.
Build and support integrations using Azure Functions and Logic Apps.
Implement and support CI/CD pipelines and promote code across Development QA Production environments using Git-based deployment processes.
Develop and support reporting/data visualization solutions using Power BI.
Troubleshoot production data issues and collaborate with business, application, and data teams.

Required Skills

Strong hands-on experience with Azure Synapse Analytics, Azure Data Factory, Azure Databricks, SQL Server, Python/PySpark, Spark, Azure Storage, Azure Key Vault, SQL Pools, Git, and CI/CD is required. Candidates should also have solid experience with stored procedures, dynamic SQL, APIs, ETL/data pipelines, delta/incremental data loads, and production troubleshooting.

Power BI experience and exposure to Azure Functions and Logic Apps are also desired.

Location requirement: Candidates must be able to work onsite in Jackson, Michigan, 3 days per week.