Please strictly adhere to the following resume naming convention:
Role: Azure Devops Databricks platform Engineer
Skills: Azure Devops
Exp: 7+ Years
Key Responsibilities
Platform & Infrastructure
Design, deploy, and manage Databricks workspaces across multiple Azure subscriptions and environments (dev, staging, production)
Implement Infrastructure-as-Code (IaC) solutions using Terraform for Databricks resources
Establish and maintain security best practices, including network isolation, RBAC, and data governance
Configure and optimize cluster management, including auto-scaling policies and cost optimization strategies
Manage Databricks workspace connectivity and networking (VNets, private endpoints, firewalls)
DevOps & Automation
Build CICD pipelines using Azure DevOps for deploying Databricks notebooks, jobs, and configurations
Implement automated testing and validation of data pipelines
Create self-service deployment frameworks and templates for development teams
Establish monitoring, alerting, and logging solutions for Databricks environments
Automate workspace provisioning and deprovisioning processes
Collaboration & Enablement
Partner with data engineers and analysts to understand platform requirements
Provide technical guidance and documentation for platform usage
Develop runbooks and operational procedures for incident response
Mentor team members on Databricks best practices and Azure DevOps tools
Governance & Compliance
Implement access controls and audit logging across Databricks workspaces
Ensure compliance with organizational security and data governance policies
Monitor and manage workspace usage, costs, and resource utilization
Document architecture decisions and maintain platform documentation
Required Qualifications
7+ years of experience in platform engineering, infrastructure engineering, or DevOps
2+ years of hands-on experience with Databricks infrastructure setup (or Apache SparkDelta Lake environments)
3+ years of experience with Azure cloud services and infrastructure
2+ years of experience with Azure DevOps (Pipelines, Repos, Artifacts)
Strong proficiency in Infrastructure-as-Code tools (Terraform Enterprise)
Solid understanding of CICD principles and implementation
Experience with LinuxUnix environments and scripting (Bash, Python, or similar)
Familiarity with containerization technologies (Docker, Kubernetes)
Strong knowledge of networking, security, and identity management in cloud environments
Preferred Qualifications
Experience with Unity Catalog and Databricks governance features
Knowledge of Apache Spark architecture and optimization
Experience with other IaC tools (Terraform)
Familiarity with cost optimization strategies in cloud environments
Azure certifications (AZ-900, AZ-104, AZ-305)
Databricks Certification
Experience with Git and version control workflows
Knowledge of data governance and data mesh architectures
Experience with monitoring tools (Datadog, New Relic, Azure Monitor)
