Databricks Platform Engineer
Locations: New York, NY; Houston, TX; Lisle, IL; Atlanta, GA; Austin, TX; Bellevue, WA; Bridgewater, NJ; Somerset, NJ; Charlotte, NC; Franklin, TN; Milwaukee, WI; Rogers, AR; San Ramon, CA; Wayne, PA.
Duration: Long term contract
This opportunity is ideal for someone who operates beyond traditional data engineering and has hands-on experience owning the Databricks platform itself. The successful candidate will establish platform standards, automate deployments, strengthen governance and security, optimize performance and cloud consumption, and provide engineering teams with a reliable foundation for enterprise analytics and data workloads.
We are looking for a hands-on engineer who understands how Databricks functions as an enterprise platform—from workspace architecture and cluster management through Unity Catalog, Infrastructure as Code, CI/CD, security, governance, observability, and cost optimization.
What You'll Be Doing
Design, configure, administer, and continuously improve enterprise Databricks environments.
Manage Databricks workspaces, clusters, compute policies, jobs, permissions, configurations, and platform services.
Establish reusable platform standards, architecture patterns, guardrails, and engineering best practices.
Own and support Unity Catalog, including catalogs, schemas, permissions, RBAC, data access policies, governance, and lineage.
Build and maintain automated infrastructure and deployment processes using Terraform and Infrastructure as Code.
Develop and support CI/CD pipelines for Databricks configurations, notebooks, jobs, workflows, and platform components.
Partner with data engineers, analytics engineers, architects, security teams, and application teams to enable scalable development.
Integrate Databricks with cloud storage, data lakes, enterprise applications, BI platforms, orchestration tools, and other data services.
Support and optimize Apache Spark workloads across large-scale distributed environments.
Monitor platform health, cluster utilization, workload performance, reliability, and cloud consumption.
Troubleshoot performance, configuration, deployment, access, and integration issues.
Automate repetitive administration and operational processes.
Implement platform controls that support enterprise security, compliance, governance, and operational standards.
Drive ongoing improvements around performance, scalability, reliability, automation, and cost optimization.
What We're Looking For Required Experience
5+ years of experience in Data Engineering, Data Platform Engineering, Cloud Platform Engineering, or Data Platform Administration.
3+ years of hands-on Databricks experience focused on administration, platform engineering, architecture, or enterprise platform ownership.
Strong hands-on experience with Azure Databricks.
Deep understanding of Apache Spark, distributed computing, workload tuning, and performance optimization.
Hands-on experience implementing and managing Unity Catalog.
Strong knowledge of:
Role-Based Access Control (RBAC)
Identity and access management
Data governance
Platform security
Permissions and access policies
Experience building infrastructure using Terraform or comparable Infrastructure-as-Code technologies.
Experience designing or supporting CI/CD pipelines and automated Databricks deployments.
Strong understanding of cloud data architecture and services within Azure; experience with AWS or Google Cloud Platform is also valuable.
Experience operating enterprise data platforms where reliability, security, scalability, and governance are critical.
Bachelor's degree in Computer Science, Engineering, Data Science, Information Technology, or a related technical discipline.
Highly Desired
Experience designing or supporting a Lakehouse Architecture.
Strong experience with Delta Lake.
Experience with Databricks Workflows, Jobs, cluster policies, SQL warehouses, or related platform capabilities.
Experience integrating Databricks with enterprise data lakes and cloud storage.
Knowledge of Structured Streaming, Kafka, Azure Event Hubs, or other streaming technologies.
Experience implementing observability, logging, monitoring, alerting, and automated operational controls.
Experience with Databricks platform performance and FinOps / cloud cost optimization.
Exposure to secrets management, service principals, private networking, or enterprise cloud security patterns.
Certifications That Stand Out
Relevant certifications are valuable but not required, including:
Databricks Certified Data Engineer Professional
Databricks Certified Data Engineer Associate
Databricks platform-related certifications
Microsoft Certified: Azure Data Engineer
Microsoft Certified: Azure Solutions Architect
Comparable AWS or Google Cloud Platform cloud certifications
Never miss an opportunity! Create an alert based on the job you applied for.