Databricks Administrator

Databricks Administrator

Location: Austin, TX (Hybrid/Onsite as required)
Type: Full-Time

Job Summary:-

We are seeking an experienced Databricks Administrator to manage and optimize Databricks workspaces, clusters, jobs, and integrations for enterprise-scale financial services workloads. This role will be responsible for ensuring the stability, performance, and security of the Databricks environment, while also driving automation and governance best practices across the organization.

Key Responsibilities:-

• Administration & Configuration: Administer, configure, and manage Databricks workspaces, clusters, jobs, and notebooks to support enterprise data and analytics workloads.
• Security & Access Management: Implement and maintain user access controls, security policies, and workspace permissions aligned with enterprise and regulatory standards.
• Performance Optimization: Monitor and optimize cluster performance, troubleshoot execution issues, and ensure efficient resource utilization and cost management.
• Integrations: Manage integrations with Azure cloud services (ADF, Synapse, Data Lake, Key Vault, etc.) and enterprise data sources such as SQL Server.
• Governance & Compliance: Ensure adherence to data governance, financial services compliance, and security best practices.
• Automation & DevOps: Automate environment provisioning, monitoring, and operational tasks using Terraform, CI/CD pipelines, or APIs.
• Collaboration: Partner with data engineering, analytics, and security teams to support business-critical workloads.
• Documentation & Standards: Develop and maintain detailed documentation, SOPs, and architectural standards for Databricks management.

Required Qualifications

• 6+ years of experience in Data Engineering / Cloud Administration, with at least 2+ years in Databricks administration.
• Strong expertise in Azure Databricks and Azure ecosystem (Azure Data Factory, Synapse, Data Lake, Key Vault, etc.).
• Hands-on experience managing Databricks clusters, jobs, notebooks, and workspace configurations.
• Proficiency in SQL, Python, and/or PySpark for troubleshooting and validation.
• Strong knowledge of identity and access management (IAM), RBAC, and workspace security policies.
• Experience with Terraform, CI/CD pipelines, and DevOps practices for automation.
• Understanding of data governance, compliance, and security controls in the financial services industry.
• Excellent problem-solving and stakeholder communication skills.

Preferred Qualifications:-

• Experience working in financial services or banking domain.
• Familiarity with Databricks Unity Catalog for governance and security.
• Azure certifications (e.g., DP-203: Azure Data Engineer Associate, AZ-305: Solutions Architect).
• Knowledge of monitoring tools such as Azure Monitor, Log Analytics, or Splunk.

Education:-

• Bachelor s or Master s degree in Computer Science, Data Engineering, Information Systems, or a related field.