Databricks Admin/Unity Catalog

Role Overview:We are seeking a highly skilled Databricks Administrator to manage our data ecosystem. In this role, you will be responsible for provisioning Databricks and Unity Catalog services, maintaining organizational governance, and automating data-related processes. You will collaborate with cross-functional teams to translate business requirements into scalable, secure data solutions.

Key Responsibilities:

• Provisioning & Architecture: Manage distributed workspaces, workspace policies, and cluster sizing/capacity. Implement data architecture leveraging Databricks, Unity Catalog, Privacera, and Collibra.
• Governance & Compliance: Ensure data security through rigorous user/group management, identity federation, and role-based access control (RBAC).
• Pipeline Optimization: Troubleshoot ETL bottlenecks and ML workloads. Develop UDFs (SQL/Python) and create data masking processes.
• Collaboration: Work alongside data scientists and analysts to operationalize ML models in Batch and Real-Time pipelines.

Required Skills & Experience:

• Core Databricks: Account/Workspace administration, configuration across environments, and compute resource optimization.
• Unity Catalog: Deep expertise in metastores, catalogs, schemas, external locations, and Delta sharing.
• Technical Proficiency: Strong scripting experience in Python and SQL.
• Security & Compliance: Proven experience with security compliance procedures and implementing guardrails for data sharing.
• Engineering Mindset: Ability to guide teams on pipeline building and performance optimization.

Preferred Qualifications:

• Understanding of Agentic Architecture.
• Familiarity with the data requirements of common ML/AI use cases.
• Experience navigating multi-step enterprise approval processes (Architecture, Security, Governance).