Azure Databricks Admin

Role Title: Azure Databricks Admin

Employment Type: Contract

Duration: 6 months to start, with chance of extension up to 18 months

Preferred Location: 4 days per week onsite in Wilmington, DE/Philadelphia PA area

Role Description:

Urgent need for an Azure Databricks admin with 5 years experience and strong Terraform skills for ongoing support onsite in Wilmington. The position is for ongoing operational support, not a short-term project.

Requirements:

Minimum 4+ years of hands on experience administering Azure Databricks platforms in an enterprise environment.

Proven experience configuring and managing Azure Databricks platform components, including:

• Clusters: autoscaling configurations, instance pools, cluster policies, and performance optimization
• Jobs and Workflows: scheduling, concurrency controls, retries, alerts, and notifications
• Workspace assets: notebooks, Git repos, libraries, init scripts, and secret scopes
• Strong experience with Unity Catalog and implementation of enterprise data governance models, including fine grained access controls and catalog level security.
• Experience with Databricks automation and deployment tooling, including:
• Databricks CLI
• Databricks REST APIs
• Databricks Asset Bundles (DAB)
• Experience integrating Databricks with BI Tools/analytics platforms.
• Solid understanding of data engineering patterns, Spark based workloads, and techniques for tuning performance and optimizing resource usage.
• Working knowledge of Microsoft Azure fundamentals, including:
• Identity and access management using RBAC, service principals, and managed identities
• Azure Data Lake Storage Gen2 (ADLS Gen2), Azure Key Vault, Azure Monitor, and Log Analytics
• Azure networking concepts such as VNets, private endpoints, and DNS resolution
• Proven expertise with Terraform, including:
• Modular design patterns
• Remote state management
• Multi environment deployment strategies
• Secure infrastructure as code practices
• Strong experience building and maintaining CI/CD pipelines using GitLab CI/CD, including:
• YAML based pipeline definitions
• Runner configuration
• Environment promotion strategies
• Approval gates and controlled deployments
• Demonstrated experience supporting compliance driven and regulated environments, including:
• Audit evidence collection and documentation
• Periodic access reviews
• Change management and release controls
• Nice to Have experience on MLOps, including:
• Experience supporting machine learning workloads on Azure Databricks
• Familiarity with MLflow for experiment tracking, model registry, and lifecycle management
• Exposure to MLOps pipelines, including model training, validation, and deployment automation
• Understanding of model governance, versioning, and promotion across environments