Platform Architect – Azure Databricks

Job Title: Platform Architect – Azure Databricks

Location: Remote – USA
Pay Rate: $60–62/hr on W2
USC and GC only
Experience: Senior Level
Industry: Retail – 3+ years of Retail experience required

Job Description

We are seeking an experienced Platform Architect with strong Azure Databricks expertise to design, secure, optimize, and manage enterprise-scale Databricks platforms. The ideal candidate will have deep experience with Azure cloud architecture, Databricks, Unity Catalog, security, networking, governance, CI/CD, and platform operations, along with 3+ years of Retail industry experience.

Key Responsibilities

• Architect and oversee Azure Databricks workspace deployments, including VNet Injection, private endpoints, secure cluster connectivity, and network security.
• Design and manage Databricks clusters, policies, Runtime versions, autoscaling, capacity, performance, and cost optimization.
• Architect and implement Unity Catalog, including catalogs, schemas, volumes, external locations, data lineage, auditing, and fine-grained access controls.
• Integrate Databricks with Azure Entra ID, SCIM, Azure Key Vault, ADLS Gen2, and enterprise security frameworks.
• Design secure Azure networking using VNets, private endpoints, NSGs, DNS, routing, VPNs, and firewalls.
• Architect Azure Data Factory (ADF) and Databricks integration and orchestration patterns.
• Develop CI/CD processes for Databricks notebooks, jobs, configurations, and infrastructure using Azure DevOps or GitHub Actions.
• Monitor and troubleshoot Databricks environments using Azure Monitor, Log Analytics, and Databricks-native monitoring.
• Support disaster recovery, business continuity, platform upgrades, and operational reliability.
• Create architecture documents, diagrams, standards, and operational runbooks.
• Provide technical leadership, knowledge transfer, and guidance to data engineering, data science, security, network, and operations teams.
• Participate in architecture reviews and serve as the Databricks platform SME.

Required Skills

• Deep expertise in Azure Databricks
• Unity Catalog, Delta Lake, Apache Spark
• Databricks Workspaces, Clusters, Jobs, Notebooks, Repos, APIs
• Azure Data Factory (ADF)
• ADLS Gen2
• Azure Key Vault
• Azure Entra ID / Azure AD and SCIM
• Azure VNets, NSGs, Private Endpoints, networking, routing, firewalls
• Azure Monitor and Log Analytics
• RBAC, IAM, ACLs, encryption, and cloud security
• Databricks Runtime and performance optimization
• Python, Scala, SQL, PowerShell/Azure CLI
• CI/CD with Azure DevOps or GitHub Actions
• Strong architecture, troubleshooting, documentation, and stakeholder-management skills
• 3+ years of Retail industry experience

Preferred Qualifications

• Microsoft Certified: Azure Solutions Architect Expert
• Databricks Certified Data Engineer certification
• Experience with enterprise-scale data platforms and cloud modernization
• Strong understanding of data governance, lineage, compliance, and platform cost optimization