Data Governance - Unity Catalog

Your Role

• Implement data provisioning patterns based on business requirements, following predefined processes, policies, standards, and metadata management rules
• Create and manage distributed workspaces in Databricks, set up workspace policies, provision Databricks clusters and manage data infrastructure sizing and capacity
• Create Python notebooks, implement data masking processes, create UDFs (SQL/Python), troubleshoot data pipelines
• Ensure data security and compliance with regulations using Databricks and Privacera's features
• Navigate multi-step enterprise approval process across architecture, security, and governance teams
• Design and implement data architecture leveraging technologies such as Databricks, Unity Catalog, and Privacera.
• Develop, optimize, and manage data pipelines for ETL processes using Databricks, with a focus on data integrity and quality
• Design and maintain data models and schemas, incorporating Unity Catalog and Collibra data governance practices
• Establish a robust data governance strategy, defining standards, metadata management, lineage, and quality practices
• Operationalize Machine Learning models in Batch and Real Time Data Pipelines, leveraging relevant governance setups
• Collaborate with cross-functional teams including data scientists, engineers, and analysts to translate business requirements into scalable solutions
• Foster collaboration and clarity in complex, ambiguous environments.

Job Description - Grade Specific

Your skills and experience:

• Extensive hands-on platform engineering experience with Databricks and Unity Catalog
• Proven success in implementing distributed Databricks workspaces, SQL, and Python, scripting, and automation
• Experience with Azure infrastructure provisioning
• Familiarity with data governance frameworks and compliance standards
• Familiarity with Privacera data security and access control management
• Familiarity with data requirements of common ML/AI use cases
• Awareness of data governance frameworks, enterprise data compliance requirements, metadata modeling, data architecture, and enterprise-scale data discovery solutions