Senior Databricks Engineer
Job Summary:
Seeking a Senior Databricks Engineer with strong experience in Databricks, Apache Spark, Delta Lake, and cloud data engineering. The candidate will design, develop, optimize, and maintain scalable data pipelines supporting analytics and AI-driven healthcare applications.
Responsibilities:
Design and develop scalable data engineering solutions using Databricks and Apache Spark.
Build and optimize ETLELT pipelines using PySpark, Python, or Scala.
Implement Delta Lake architecture and distributed data processing solutions.
Optimize Spark jobs, Databricks clusters, pipelines, performance, and cloud costs.
Develop CICD, testing, monitoring, and deployment processes for Databricks workloads.
Troubleshoot complex data pipeline and Spark performance issues and perform root-cause analysis.
Support ML model deployment and lifecycle management within Databricks.
Collaborate with data engineers, data scientists, architects, and business stakeholders in Agile environments.
Required Skills:
Databricks
Apache Spark PySpark
Delta Lake
Python or Scala
ETLELT & Data Engineering
AzureAWSGoogle Cloud Platform
Preferred: Azure Databricks, MLflow, Spark SQL, CICD, Azure DevOps, Terraform, Machine Learning, Data Governance, Healthcare Analytics.
Education: Bachelors degree in Computer Science, Engineering, IT, or related field preferred.
Never miss an opportunity! Create an alert based on the job you applied for.