Azure Databricks Engineer

Overview

Remote

$130,000 - $160,000

Full Time

Unable to Provide Sponsorship

Skills

data bricks

unity catalog

azure

Job Details

Job Title: Azure Databricks Engineer (Unity Catalog) Remote
Location: Remote (U.S. Based)

Job Type: Full-Time

Visa Status: Open to all visa types (H1B, etc.)

Job Overview:

We are seeking a highly skilled and experienced Azure Databricks Engineer with hands-on expertise in Unity Catalog to support the development and governance of our modern data platform. This is a fully remote role offering flexibility and the opportunity to work with cutting-edge cloud data technologies in a collaborative environment.

Key Responsibilities:

Design, build, and optimize large-scale data processing solutions using Azure Databricks.
Implement and manage Unity Catalog for fine-grained data access control, lineage tracking, and governance across workspaces.
Define and enforce data governance, security models, and access policies using Unity Catalog.
Integrate Azure Databricks with Azure Data Lake Storage Gen2, Data Factory, and other Azure services.
Collaborate with cross-functional teams including data architects, analysts, and DevOps engineers to deliver scalable data solutions.
Monitor, debug, and fine-tune Spark jobs to ensure performance and reliability.
Support CI/CD deployment processes and automate workflows where possible.
Maintain documentation and provide knowledge-sharing sessions on best practices for Unity Catalog and data security.

Required Qualifications:

3+ years of experience with Azure Databricks and data engineering in cloud environments.
1+ years of hands-on experience with Unity Catalog in production settings.
Strong skills in PySpark, SQL, and Delta Lake.
Solid understanding of Azure data ecosystem: Azure Data Lake, Data Factory, Synapse, Key Vault, etc.
Experience with RBAC, ABAC, and managing access controls within Unity Catalog.
Familiarity with CI/CD pipelines and infrastructure-as-code tools like Terraform or Azure DevOps.
Excellent problem-solving and communication skills.

Nice to Have:

Databricks or Microsoft Azure certifications (e.g., Databricks Certified Data Engineer).
Experience with audit logging, lineage tracking, or compliance frameworks.
Background in industries such as finance, healthcare, or e-commerce.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.