Lead Azure Data Engineer with AI/ML exp - Atlanta, GA and Chicago, IL (Onsite)

Overview

On Site
Depends on Experience
Contract - Independent
Contract - W2
Contract - 6 month(s)
No Travel Required

Skills

Big data
Python
data engineer
ETL
Databricks
Azure Data Factory
Azure devops
GenAI
RAG
Synapse Pipelines
Azure Machine Learning
Azure OpenAI

Job Details

Role Name: Lead Azure Data Engineer
Duration: 6 Months
Location: Atlanta, GA and Chicago, IL (Onsite)

Key Responsibilities:
• Design, build, and maintain scalable ETL/ELT pipelines using Azure Data Factory, Synapse Pipelines, and Databricks.
• Ingest, transform, and curate structured and unstructured data from diverse sources for AI/ML consumption.
• Optimize data workflows for performance, reliability, and cost-efficiency.
• Enable data access and preparation for Azure AI services including Azure Machine Learning, Azure OpenAI, and Cognitive Services.
• Collaborate with data scientists and ML engineers to operationalize AI models using MLOps and GenAIOps practices.
• Support RAG (Retrieval-Augmented Generation) and other GenAI patterns through data engineering best practices.
• Implement CI/CD pipelines for data workflows using Azure DevOps or GitHub Actions.
• Automate infrastructure provisioning using Terraform, Bicep, or ARM templates.
• Monitor and troubleshoot data pipelines and cloud resources using Azure Monitor and Log Analytics.

Skills and Qualifications:
• 8- 10+ years of experience in data engineering, cloud platforms, and big data technologies.
• Strong proficiency in Azure services: Data Factory, Synapse, Data Lake, Azure SQL, Cosmos DB.
• Experience with Python, Spark, SQL, and distributed data processing.
• Familiarity with AI/ML workflows and tools such as Azure ML, MLflow, and OpenAI APIs.
• Understanding of data governance, security, and compliance in cloud environments.

 

 

Best Regards,

Vishal

Truth Lies in  Heart

 

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.