Job Description – Azure Data Engineer
Location: Louisville, KY (5 Days Onsite)
Mandatory Skills: Azure Databricks, PySpark
OPT visa will work .
Position Summary
We are seeking an experienced Azure Data Engineer with strong expertise in Azure Databricks & PySpark to design, develop, optimize, and maintain large-scale data pipelines for healthcare domain projects at Humana. The engineer will work closely with data architects and business stakeholders to ensure high-quality, scalable, and efficient data solutions on Azure Cloud.
Key Responsibilities
• Develop and maintain ETL/ELT pipelines using Azure Databricks and PySpark.
• Perform data ingestion, transformation, and validation from multiple sources into Azure Data Lake.
• Implement Delta Lake architecture (Bronze, Silver, Gold layers).
• Optimize PySpark code, cluster configurations, and Spark jobs for performance and cost efficiency.
• Design data workflows using Azure Data Factory (ADF) and integrate with Databricks.
• Work with Azure services such as ADLS Gen2, Azure SQL, Synapse, Key Vault, and Azure DevOps.
• Troubleshoot data pipeline issues and ensure high availability and reliability.
• Maintain documentation, data dictionaries, and technical design artifacts.
• Ensure data security, compliance, and quality standards (especially relevant to healthcare).
• Collaborate with cross-functional teams including Architects, QA, Business Analysts, and Product Owners.
Required Skills & Experience
• 3+ years of experience as a Data Engineer.
• Strong hands-on experience with Azure Databricks and PySpark.
• Solid understanding of Spark SQL, Delta Lake, and distributed data processing.
• Experience working with Azure Data Factory, ADLS Gen2, and Azure Cloud ecosystem.
• Proficient in ETL/ELT frameworks, data modeling, and performance tuning.
• Strong SQL skills for data analysis and transformations.
• Experience working with CI/CD pipelines using Azure DevOps or Git.
• Good understanding of healthcare data (HIPAA, claims, provider/member data) is a plus.
Never miss an opportunity! Create an alert based on the job you applied for.
