Position: Senior Azure Databricks Platform Engineer
Location: Remote
Duration: 6-12 months
Domain Focus: Databricks, Azure Cloud, Enterprise Data Platforms, Platform Engineering, DevOps Enablement
Key Responsibilities:
Databricks Platform Engineering & Operations
· Support architecture and operational engineering across enterprise Databricks environments
· Design and maintain scalable Lakehouse, Delta Lake, and Unity Catalog solutions
Support administration of:
· Databricks Workspaces
· Jobs
· Notebooks
· Clusters
· Optimize Spark workloads, cluster configurations, and platform performance
· Support batch and streaming data processing workloads
· Assist with multi-environment platform management across Dev, QA, and Production
· Platform Integration & DevOps Alignment
· Support integration of Databricks platforms with Azure-native ecosystems and enterprise applications
Work across integrations involving:
· APIs
· Azure Storage Accounts
· Data Lakes
· reporting and analytics platforms
· Collaborate with platform and DevOps teams to align Databricks activities with existing CI/CD and automation standards
Support deployment workflows utilizing:
· GitHub
· Azure DevOps
· Infrastructure as Code frameworks
· Help ensure modernization activities do not disrupt existing automated deployment pipelines and operational processes
Governance, Reliability & Platform Support:
Support governance, access management, monitoring, logging, and operational security practices
Assist with implementation of:
· RBAC
· lineage
· encryption
· platform governance controls
· Participate in troubleshooting, operational support, and root cause analysis activities
· Contribute to documentation, operational standards, and platform best practices
Required Qualifications
· 6+ years of experience within cloud, platform engineering, data engineering, or enterprise analytics ecosystems
· 3+ years of hands-on experience supporting enterprise Databricks environments
· Strong experience with:
o Delta Lake
o Unity Catalog
o Spark workloads
o cluster operations
o Databricks Workspaces
o Experience with Azure cloud services and cloud-native platform concepts
Familiarity with:
o Azure DevOps
o GitHub workflows
o CI/CD pipeline concepts
o deployment automation
o Experience supporting APIs, distributed application integrations, and enterprise data platforms
o Strong troubleshooting and operational support capabilities
o Strong communication and stakeholder collaboration skills
Preferred Qualifications
Experience with:
· PySpark
· Python
· SQL
· Spark SQL
· Scala
Familiarity with:
o AKS/Kubernetes
o Terraform
o Bicep
o Infrastructure as Code practices
o Azure Data Lake
o Event Hub
o streaming architectures
o enterprise analytics environments
Familiarity with monitoring and observability platforms such as:
o Azure Monitor
o Datadog
o Exposure to enterprise modernization and cloud migration initiatives
Never miss an opportunity! Create an alert based on the job you applied for.
