Databricks Architect- AWS (Healthcare Domain)

Databricks Architect AWS (Healthcare Domain)
Location: Remote
Contract role

Experience

10+ years in data engineering/architecture, 3 5+ years on Databricks and AWS

Role Overview

We are seeking an experienced Databricks Architect with strong expertise in AWS cloud and a proven background in the Healthcare domain. The ideal candidate will design, architect, and lead scalable analytics and data engineering solutions leveraging Databricks, Spark, and AWS services while ensuring compliance with healthcare regulations such as HIPAA.

Key Responsibilities

• Design and architect end-to-end Databricks-based data platforms on AWS for large-scale healthcare data.
• Lead the development of Lakehouse architectures using Databricks (Delta Lake, Unity Catalog).
• Define data ingestion patterns for structured and unstructured healthcare data (EHR, claims, HL7, FHIR, imaging, IoT).
• Architect real-time and batch data pipelines using Apache Spark, Delta Live Tables, and Databricks Workflows.
• Implement data governance, security, and access controls aligned with HIPAA and organizational policies.
• Optimize performance, scalability, and cost for Databricks workloads on AWS.
• Collaborate with data scientists, analysts, compliance teams, and business stakeholders.
• Define best practices, reference architectures, and coding standards.
• Lead and mentor data engineers and contribute to architectural decision-making forums.
• Support CI/CD, version control, and DevOps practices for Databricks (Databricks Repos, Git integration).

Required Technical Skills

Databricks & Big Data

• Databricks on AWS (clusters, jobs, notebooks, workflows)
• Apache Spark (PySpark / Spark SQL)
• Delta Lake, Unity Catalog
• Data modeling for analytics (star/snowflake schemas)
• Real-time and batch processing patterns

AWS Cloud

• S3, EC2, IAM, VPC
• AWS Glue, Athena, Redshift
• AWS Lambda, Step Functions
• CloudWatch, CloudTrail
• Security and encryption (KMS, IAM policies)

Data Engineering & Integration

• ETL/ELT design and optimization
• Streaming technologies (Kafka, Kinesis)
• APIs and data integration
• SQL and Python (strong proficiency)

Healthcare Domain Experience (Mandatory)

• Strong understanding of Healthcare data models and workflows
• Experience with EHR, EMR, claims, payer/provider data
• Familiarity with HL7, FHIR, ICD-10, CPT, SNOMED
• HIPAA compliance, PHI/PII handling, data masking
• Experience with quality measures, clinical analytics, or population health is a plus

Preferred Qualifications

• Databricks Certified Data Engineer / Databricks Certified Architect
• AWS Certified Solutions Architect (Associate or Professional)
• Experience with MLOps or MLflow in Databricks
• Familiarity with data visualization tools (Power BI, Tableau, Looker)
• Experience in large healthcare enterprises or HealthTech companies

Education

• Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or related field

Soft Skills

• Strong communication and stakeholder management
• Ability to translate business and clinical requirements into technical solutions
• Leadership mindset and mentoring capability
• Problem-solving and decision-making skills