Databricks Adminisrator

Databricks Platform Admin

Rariton, NJ (Onsite)

Contract

Responsibilities:

Job Description:

• JD - Built Databricks Platform environments from scratch across AWS and Azure clouds, including creating VPCx accounts, configuring networking, and setting up secure storage in AWS S3 (with external locations) and Azure ADLS Gen2 containers.
• Design the Databricks security model and manage user/group access through Active Directory (AD) groups, ensuring the right permissions for catalogs, schemas, workspaces, and related resources.
• Design and implement Unity Catalog structures (bronze, silver, gold layers) with associated schemas, following governance standards for data organization and access control.
• Create and manage All-Purpose clusters and SQL Warehouses, applying cluster policies, attaching libraries, and tagging resources for tracking in FinOps billing dashboards.
• Create secret scopes for storing sensitive credentials securely, making sure no passwords or keys are hard-coded in notebooks or workspaces.
• Support teams in resolving job/workflow issues and failures.
• Help teams set up MBox for file transfers from storage accounts to Databricks volumes.
• Develop and maintain CI/CD automation using Terraform and Jenkins (service account) to create repo folders, catalogs, schemas, external locations, All-Purpose clusters, and SQL Warehouses — removing the need for manual setup.
• Configure secure connections to enterprise data sources like SAP, Teradata, and PostgreSQL, including driver installation, secret setup, and private endpoint creation for SQL Warehouse serverless usage.
• Implemented Lakehouse Federation to query external databases directly from Databricks and enable Delta Sharing for cross-workspace and cross-cloud data exchange between AWS and Azure, including catalog binding for governance.
• Work with business owners during onboarding and collaborate with application owners and developers to solve platform issues; actively join scrum calls for 8–10 supported teams.
• Prepare and present PPTs on platform enablement activities to business owners and maintain up-to-date documentation in CBT Confluence () on platform capabilities, configurations, and processes.
• Monitor and optimize compute usage for cost savings, apply tagging for detailed billing insights, and ensure all activities follow JnJ governance and change-management processes.