Databricks Engineer

Overview

We are seeking an experienced Senior Databricks Consultant with a minimum of 10+ years in data engineering and consulting, including extensive hands-on expertise with Databricks Unity Catalog implementation and migration. This role will lead enterprise clients through the modernization of their data governance, security, and cataloging capabilities, ensuring a smooth transition to Unity Catalog while aligning with compliance and business objectives.

Key Responsibilities

Implementation & Migration

• Architect, implement, and oversee Databricks Unity Catalog deployments across enterprise-scale environments.
• Lead end-to-end migrations from legacy Hive Metastore, Delta Lake, or third-party catalogs into Unity Catalog.
• Design and enforce fine-grained access control models, lineage tracking, and data governance standards.

Consulting & Best Practices

• Serve as a trusted advisor to client executives and technical leaders on data governance strategy and Unity Catalog adoption.
• Develop best practices for identity and access management (IAM), compliance, data sharing, and multi-cloud deployments.
• Provide solution design guidance, ensuring alignment with enterprise security, governance, and compliance frameworks.

Enablement & Leadership

• Create migration frameworks, documentation, and reusable accelerators to standardize Unity Catalog adoption.
• Conduct workshops and training sessions for client teams to build long-term capabilities.
• Mentor junior engineers and consultants, setting technical standards and reviewing deliverables.

Qualifications

• Education: Bachelor s or Master s in Computer Science, Data Engineering, Information Systems, or related field.
• Experience:
• 10+ years in data engineering, data platforms, or consulting roles.
• 5+ years of Databricks experience, including advanced use of Delta Lake, Spark, and SQL.
• Proven experience leading enterprise-scale Unity Catalog implementations and migrations.
• Skills:
• Expertise in Python, SQL, Spark, and Databricks APIs.
• Strong understanding of Azure, AWS, or Google Cloud Platform data platform architectures.
• Deep knowledge of data governance, lineage, and enterprise security practices.
• Excellent consulting skills: stakeholder engagement, requirements analysis, and solution delivery.