Senior Databricks Engineer
Remote
Position Summary
The Senior Data Engineer — Databricks will own the end-to-end data lifecycle, from analysis and solution design through implementation, orchestration, governance, and optimization. This role will build modern data solutions on the Databricks Data Intelligence Platform across Microsoft Azure, AWS, or Google Cloud Platform, using Databricks, Python, PySpark, and SQL to deliver scalable ingestion, transformation, modeling, and pipeline orchestration.
As a senior engineer, you will lead solution design, establish engineering standards, and guide other data engineers in building reliable, governed, and production-ready data pipelines that support analytics, reporting, and AI/ML use cases.
Qualifications
- 10+ years of professional experience in data engineering, data and analytics, or data warehousing.
5+ years of hands-on experience working with Databricks in a data engineering capacity, including production workloads.
5+ years of experience with Python, including manipulating large datasets using PySpark DataFrames, Spark SQL, and/or the pandas API on Spark.
3+ years of data engineering experience on Microsoft Azure, including services such as ADLS Gen2, Azure Data Factory, Key Vault, and Microsoft Entra ID.
Strong experience designing and developing ETL/ELT pipelines for batch and streaming data workloads.
Experience with Databricks lakehouse architecture, Delta tables, Unity Catalog, Lakeflow Jobs, and Lakeflow Spark Declarative Pipelines.
Experience implementing scalable data models using medallion architecture, dimensional modeling, and slowly changing dimensions.
Experience with CI/CD and testing practices for data pipelines using Azure DevOps or GitHub Actions.
Ability to lead technical design discussions, conduct code reviews, define engineering standards, and mentor other engineers.
Strong collaboration skills with the ability to work across business, IT, and technical stakeholder groups.
Preferred certification: Databricks Data Engineer Professional.
