Data Engineer (Databricks | Healthcare Analytics)

Data Engineer (Databricks | Healthcare Analytics)

The Data Engineer is responsible for designing, building, and maintaining scalable data pipelines and data platforms that support enterprise analytics, business intelligence, and clinical reporting. This role plays a critical part in enabling data-driven decision-making by integrating data from Electronic Medical Record (EMR) systems and other healthcare sources into a modern data ecosystem.

The ideal candidate has strong experience with Databricks, SQL, and Python, along with a deep understanding of healthcare data, EMR systems, and analytics workflows.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks to process large and complex healthcare datasets.
• Build and optimize ETL/ELT workflows to ingest, transform, and load data from EMR systems (e.g., Epic, Cerner) and other clinical, operational, and financial sources.
• Develop and manage data models to support enterprise reporting, analytics, and BI solutions.
• Collaborate with Data Analysts, BI developers, and business stakeholders to understand data requirements and deliver high-quality datasets.
• Ensure data quality, integrity, and consistency through validation, monitoring, and error-handling mechanisms.
• Implement data governance, security, and compliance standards aligned with healthcare regulations (e.g., HIPAA).
• Optimize data processing performance and cost efficiency within the Databricks environment.
• Develop reusable frameworks, notebooks, and workflows within Databricks for scalable data engineering and analytics use cases.
• Integrate data pipelines with cloud platforms and enterprise data warehouses.
• Support BI and analytics teams by delivering curated, analytics-ready datasets for tools such as Power BI, Tableau, or Looker.
• Monitor and troubleshoot data pipelines and production jobs, ensuring high availability and reliability.
• Document data architecture, data flows, transformation logic, and pipeline processes.
• Stay current with emerging technologies and best practices in data engineering, healthcare data, and analytics.

Required Qualifications

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field (or equivalent experience).
• 3+ years of experience in data engineering or related roles.
• Strong hands-on experience with Databricks.
• Advanced proficiency in SQL and strong programming skills in Python.
• Experience with Power BI
• Experience building and maintaining ETL/ELT pipelines in a cloud-based environment.
• Experience working with healthcare data and EMR/EHR systems (e.g., Epic, Cerner).
• Strong understanding of data warehousing concepts and dimensional modeling.
• Experience supporting BI and analytics workloads.

Preferred Qualifications

• Experience with cloud platforms such as Azure, AWS, or Google Cloud Platform.
• Familiarity with healthcare data standards (HL7, FHIR).
• Experience with data orchestration tools (e.g., Airflow, Azure Data Factory).
• Knowledge of data governance, lineage, and metadata management tools.
• Experience optimizing Spark jobs and large-scale data processing.
• Exposure to real-time or streaming data pipelines.