Senior Databricks Platform Engineer

Role: Senior Databricks Platform Engineer - Azure Cloud & Data Operations



100% Remote



Long Term Contract

Tax Term: W2



Domain Focus: Databricks, Azure Cloud, Enterprise Data Platforms, Platform Engineering, DevOps Enablement






About the Opportunity



Our client is undergoing a large-scale enterprise modernization initiative focused on Azure-native infrastructure, platform engineering, automation, and modernization of data-intensive application ecosystems.



As part of this transformation, multiple calculation-heavy and operationally critical applications are being integrated into a growing Databricks-centered enterprise data platform spanning:





Azure cloud services

AKS and containerized platforms

GitHub and CI/CD ecosystems

enterprise automation and analytics environments





We are seeking a Senior Databricks Platform Engineer to support architecture, operational engineering, platform enablement, and modernization activities across this evolving environment.



This role blends:

Databricks platform expertise

cloud infrastructure awareness

DevOps-aligned engineering

operational platform support





The ideal candidate understands how Databricks workloads, APIs, pipelines, storage layers, and operational processes integrate within broader enterprise cloud ecosystems.



Role Overview



The Senior Databricks Platform Engineer will support the design, operational enablement, modernization, and ongoing support of enterprise Databricks platforms and associated data processing environments.



This engineer will work closely with:





cloud platform teams

DevOps organizations

infrastructure engineering teams

application modernization groups

data and analytics stakeholders





The role requires strong awareness of how Databricks modernization activities impact:





existing CI/CD pipelines

automation frameworks

operational governance

enterprise cloud platform standards







Key Responsibilities

Databricks Platform Engineering & Operations

Support architecture and operational engineering across enterprise Databricks environments

Design and maintain scalable Lakehouse, Delta Lake, and Unity Catalog solutions





Support administration of:

Databricks Workspaces

Jobs

Notebooks

Clusters





Optimize Spark workloads, cluster configurations, and platform performance

Support batch and streaming data processing workloads

Assist with multi-environment platform management across Dev, QA, and Production





Platform Integration & DevOps Alignment





Support integration of Databricks platforms with Azure-native ecosystems and enterprise applications

Work across integrations involving:



APIs

Azure Storage Accounts

Data Lakes

reporting and analytics platforms





Collaborate with platform and DevOps teams to align Databricks activities with existing CI/CD and automation standards

Support deployment workflows utilizing:



GitHub

Azure DevOps

Infrastructure as Code frameworks





Help ensure modernization activities do not disrupt existing automated deployment pipelines and operational processes





Governance, Reliability & Platform Support





Support governance, access management, monitoring, logging, and operational security practices

Assist with implementation of:



RBAC

lineage

encryption

platform governance controls





Participate in troubleshooting, operational support, and root cause analysis activities

Contribute to documentation, operational standards, and platform best practices





Required Qualifications

6+ years of experience within cloud, platform engineering, data engineering, or enterprise analytics ecosystems

3+ years of hands-on experience supporting enterprise Databricks environments

Strong experience with:



Delta Lake

Unity Catalog

Spark workloads

cluster operations

Databricks Workspaces





Experience with Azure cloud services and cloud-native platform concepts

Familiarity with:



Azure DevOps

GitHub workflows

CI/CD pipeline concepts

deployment automation





Experience supporting APIs, distributed application integrations, and enterprise data platforms

Strong troubleshooting and operational support capabilities

Strong communication and stakeholder collaboration skills





Preferred Qualifications



Experience with:

PySpark

Python

SQL

Spark SQL

Scala





Familiarity with:



AKS/Kubernetes

Terraform

Bicep

Infrastructure as Code practices





Experience with:



Azure Data Lake

Event Hub

streaming architectures

enterprise analytics environments





Familiarity with monitoring and observability platforms such as:



Azure Monitor

Datadog





Exposure to enterprise modernization and cloud migration initiatives





What This Role Is

A hands-on enterprise Databricks platform engineering role

A modernization-focused engineering opportunity within a large Azure ecosystem

A hybrid role spanning cloud integration, operational engineering, and DevOps-aware platform support

A collaborative engineering role supporting scalable enterprise data and analytics platforms





What This Role Is Not

A pure data scientist or analytics-only role

A narrowly scoped ETL development position