Senior Databricks Developer
Location :Remote/Hybrid
Job Summary
We are seeking a Senior Databricks Developer with strong expertise in Databricks, Python, and cloud-based data engineering, preferably within the healthcare domain. The ideal candidate will have extensive experience building scalable data pipelines, developing cloud-native analytics solutions, and working in Agile delivery environments. Experience with AWS and healthcare data platforms is highly desirable.
Key Responsibilities
Design, develop, and optimize large-scale data processing solutions using Databricks.
Build and maintain robust ETL/ELT pipelines using Python and Databricks.
Develop scalable data ingestion, transformation, and integration frameworks from multiple data sources.
Leverage Databricks features including Delta Lake, Spark SQL, Unity Catalog, Workflows, and ML capabilities.
Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
Work with AWS services to support cloud-based data platforms and analytics workloads.
Implement data quality checks, monitoring, performance tuning, and operational support processes.
Participate in architecture discussions and contribute to best practices for data engineering and cloud adoption.
Ensure compliance with security, governance, and data management standards.
Actively participate in Agile ceremonies including sprint planning, backlog grooming, daily standups, sprint reviews, and retrospectives.
Required Skills & Qualifications
Mandatory Skills
7+ years of overall IT experience in Data Engineering and Analytics.
Strong hands-on experience with Databricks development and administration.
Strong Python programming skills for data processing, automation, and analytics.
Experience with Apache Spark (PySpark) and distributed data processing.
Strong experience in designing and developing ETL/ELT pipelines.
Experience working with structured and unstructured data.
Hands-on experience with Git version control and CI/CD practices.
Strong understanding of data modeling, data warehousing, and performance optimization.
Mandatory experience working in Agile/Scrum delivery models.
Excellent analytical, problem-solving, and communication skills.
Preferred Skills
Working knowledge of AWS services such as:
o S3
o Glue
o Lambda
o EC2
o IAM
o CloudWatch
o Redshift (preferred)
Experience with healthcare data ecosystems.
Knowledge of FHIR, HL7, Claims, Membership, Provider, or Clinical data domains is an advantage.
Exposure to data governance, data quality, and security frameworks.
Preferred Experience
Healthcare/Payer/Provider industry experience is a strong advantage.
Experience supporting enterprise-scale cloud data platforms.
Exposure to DevOps and DataOps practices.
Experience mentoring junior developers and leading technical workstreams.
Education
Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field.
Nice to Have
Databricks Certifications.
AWS Cloud Certifications.
Healthcare domain certifications or relevant project experience.
Never miss an opportunity! Create an alert based on the job you applied for.