Azure Databricks Platform Administrator

  • Memphis, TN
  • Posted 4 days ago | Updated 4 days ago

Overview

On Site
$140,000 - $160,000
Full Time

Skills

Azure

Job Details

Job Title: Azure Databricks Platform Administrator

Location: Memphis, TN (or) Maryville, TN

Full Time position

Onsite position

Job description:

Client seeking to hire an experienced Azure Databricks Platform Administrator to join our talented Information Technology team. The position offers growth, flexibility, and a collaborative work environment. The Azure Databricks Platform Administrator oversees the design, implementation, and maintenance of Databricks clusters, ensuring scalability, performance, and security. They collaborate with leadership to align platform capabilities with business goals and provide escalated support to users. Responsibilities also include enforcing policies, overseeing access controls, and automating admin tasks. In this position you should have the following: excellent interpersonal skills with the ability to communicate at all levels. Strong problem solving and creative skills and the ability to exercise sound judgment. Most important, demonstrate a high level of integrity and dependability with a strong sense of urgency and results-orientation.

This role oversees the entire Databricks infrastructure, ensuring its scalability, performance, and security. They lead a team of administrators and align platform capabilities with business goals.

  • Strong experience with Azure, Databricks and the enterprise tools they interact with (e.g Salesforce, Informatica)
  • Oversee the design, implementation, and maintenance of Databricks clusters.
  • Ensure the platform s scalability, performance, and security.
  • Develop and enforce policies and procedures for platform administration.
  • Collaborate with leadership to align platform capabilities with business goals.
  • Provide escalated support and troubleshooting to users.
  • Oversee maintenance of role-based access to data and features in the Databricks Platform using Unity Catalog.
  • Oversee implementation of external access controls for outside teams using service principles, SQL warehouses and Delta Sharing
  • Knowledge of Databricks development, including Azure Storage Technologies
  • Support Application integration and security management
  • Azure Cloud with Network security
  • Excellent verbal/written communication, collaboration, analytical and presentation skills Job responsibility
  • Provide operational support as Azure Databricks admin
  • Responsible for user provisioning, access controls and group administration
  • Manage - Databricks clusters and permissions- Azure Data Storage, key vault, Tokens, and secrets- Code deployment to production- Linked services, SQL endpoints
  • Responsible for clusters health and Performance management
  • Continuously improve processes and systems used to manage infrastructure, users, and external access.
  • Lead implementation of service principles and access tokens for external users.
  • Oversee creation of queries and dashboards to monitor critical systems and processes.
  • Review and maintain documentation for users and administrators.
  • Automate common admin tasks using Databricks notebooks.
  • Manage workflow tags, cluster tags, workflow naming convention enforcement.
  • Review clusters health check and best practices implementation.
  • Ensure regular backup and recovery.
  • Oversee privilege reviews on users and resources.
  • Stay current with industry trends and advancements in Databricks and cloud technologies.
  • Lead and mentor a team of Databricks administrators.
  • Provide training to junior members of the team and act as a point of escalation for complex questions and issues.

Experience and Education:

  • 9 years of relevant experience required.
    • Experience with Azure Databricks preferred.
  • Bachelor s degree in related field required (combination of education and experience may be considered in lieu of degree)

Technical Skills:

  • Strong understanding of data engineering needs
  • Strong understanding of Azure infrastructure required by Databricks
  • Experience with compliance audits/audit documentation
  • Experience using Databricks APIs with tools like Postman
  • Understanding of the Databricks workspace and development environment
  • Experience using Bitbucket to manage source code, and Understand Azure DevOps, managed code deployment and knowledge in CI/CD pipelines
  • Strong SQL skills for creating admin related queries and dashboards
  • Strong debugging skills
  • Ability to do risk analysis for 3rd party integrations with Azure resources.
  • Familiarity with Dynatrace.
  • Proficiency in Spark, Python, SQL, and cloud platforms (AWS, Azure, or Google Cloud Platform).
  • Strong knowledge with Python, PowerShell, pySpark
  • Strong in Python and PowerShell scripting.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.