Must-Have Experience
• 8+ years in Data Engineering, Data Architecture, or Data Platform roles
• 3+ years hands-on Databricks (Lakehouse, Delta Lake, Unity Catalog, PySpark)
• Medallion Architecture implementation (Bronze/Silver/Gold)
• Unity Catalog / data governance rollout experience
• Enterprise data governance — Role based access control, metadata management, data classification, lineage, access control
• Scalable ingestion, transformation, and provisioning pipelines
• Azure (ADF, DevOps, Synapse) and/or AWS (S3, Lambda, Glue, Redshift, MWAA)
• Python and PySpark development
Highly Desired
• Test Data Management (data masking, de-identification, synthetic data generation)
• AI-powered data discovery and classification
• Generative AI / Databricks Mosaic AI
• Data observability and monitoring frameworks
• Cross-team framework rollout and enablement
Nice to Have
• CI/CD and IaC (Terraform, Azure DevOps, GitHub Actions)
• Regulatory compliance (PII, GDPR, CCPA, HIPAA)
Key Responsibilities
• Roll out TDM framework across projects on Databricks
• Establish governance controls for sensitive data in lower environments
• Build scalable pipelines for test data generation, masking, and refresh
• Implement metadata management and lineage capabilities
• Design synthetic data strategies for non-production environments
• Enable self-service access to compliant test datasets
• Develop reusable, automated data provisioning and lifecycle frameworks
• Partner with QA, application teams, data governance, and security
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: 91140717
- Position Id: 9140130
- Posted 6 hours ago
