Data Bricks / Data Engineer

Job Summary

We are looking for a Senior Databricks Developer with strong healthcare domain experience to join our data engineering team. The ideal candidate will have deep expertise in Databricks, Apache Spark, and cloud platforms (Azure), along with a solid understanding of healthcare data standards such as HL7, FHIR, HIPAA compliance, and payer/provider data models. This role involves designing and implementing scalable data pipelines, enabling advanced analytics, and ensuring compliance with healthcare regulatory frameworks.

Key Responsibilities

• Design, build, and optimize data pipelines and ETL processes on Databricks for healthcare data integration.
• Work with healthcare datasets (EHR/EMR, claims, provider, member, clinical data) and ensure compliance with HIPAA and PHI security standards is a HUGE PLUS
• Collaborate with data scientists, analysts, and healthcare stakeholders to deliver data solutions for population health management, patient outcomes, and operational efficiency.
• Develop and maintain Delta Lake/Lakehouse architectures tailored for healthcare data storage and analytics.
• Optimize Databricks clusters, job scheduling, and performance for large-scale healthcare datasets.
• Ensure data quality, governance, and lineage using tools such as Unity Catalog or Purview.
• Support healthcare interoperability standards (HL7, FHIR, CCD, X12 transactions).
• Contribute to cloud migration and modernization projects within healthcare systems (Azure)
• Mentor junior developers and guide best practices in healthcare data engineering.

Required Skills & Qualifications

• Bachelor s or Master s degree in Computer Science, Data Engineering, or related field.
• 7+ years of experience in data engineering with at least 3+ years of hands-on experience in Databricks (Azure/AWS Databricks preferred).
• Strong proficiency in Apache Spark (PySpark/Scala/SQL).
• Proven experience with healthcare data sets (claims, EHR/EMR, clinical, payer-provider integration).
• Knowledge of healthcare interoperability standards (HL7, FHIR, X12, HIPAA compliance).
• Expertise in ETL/ELT workflows and orchestration (ADF, Airflow, Glue, etc.).
• Strong skills in SQL optimization, data modeling, and performance tuning.
• Hands-on experience with cloud platforms (Azure Data Lake)
• Familiarity with data governance frameworks (Unity Catalog, Collibra, Purview).
• Excellent communication and stakeholder management skills in healthcare environments.

Preferred Qualifications

• Experience with healthcare analytics use cases (quality reporting, risk adjustment, population health, clinical outcomes).
• Exposure to BI tools such as Power BI or Tableau for healthcare dashboards.
• Certification in Databricks, Azure is a PLUS .
• Experience in MLOps or machine learning pipelines within healthcare is a Plus