Overview
Remote
Depends on Experience
Full Time
Skills
ARM
Apache Kafka
Bash
Cloud Computing
Databricks
GRID
DevOps
Continuous Delivery
Orchestration
Python
Scripting
Terraform
Microsoft Azure
Microservices
Analytics
Grafana
GitHub
Data Processing
Data Governance
Optimization
Computer Networking
Job Details
Title: Azure Databricks DevOps Lead
Location: Remote - Full-Time
Required Skills & Experience:
Overall 9+ years of D&A experience
5+ years of relevant experience in DevOps
Strong hands-on experience with Azure Databricks, Delta Lake, and Apache Spark.
Deep understanding of Azure services: Resource Manager, AKS, ACR, Key Vault, and Networking.
Proven experience in microservices architecture and container orchestration.
Expertise in infrastructure-as-code, scripting (Python, Bash), and DevOps tooling.
Familiarity with data governance, security, and cost optimization in cloud environments.
Activities:
Architect, configure, and optimize Databricks Pipelines for large-scale data processing within an Azure Data Lakehouse environment.
Set up and manage Azure infrastructure components including Databricks Workspaces, Azure Containers (AKS/ACI), Storage Accounts, and Networking.
Design and implement a monitoring and observability framework using tools like Azure Monitor, Log Analytics, and PrometheGrafana.
Collaborate with platform and data engineering teams to enable microservices-based architecture for scalable and modular data solutions.
Drive automation and CI/CD practices using Terraform, ARM templates, and GitHub Actions/Azure DevOps.
Bonus:
Experience with event-driven architectures (Kafka/Event Grid).
Knowledge of data mesh principles and distributed data ownership.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.