Databricks Architect

Responsibilities:

• Define and maintain reference architectures (Lakehouse, CDC, streaming) and domain data models (conceptual, logical, physical).
• Create and enforce data standards: naming conventions, data types, modeling practices, semantic definitions (aligned to business glossaries).
• Establish metadata operating model: ownership, stewardship, processes for Catalog, Glossary, Data Dictionary, and Data Lineage.
• Integrate lineage capture across pipelines (ETL/ELT/streaming), BI layers, and ML workflows.
• Architect cross-platform data flows across Databricks, Oracle/SQL Server, Snowflake and metadata tools.
• Define IAM models: RBAC/ABAC, SSO/federation, SCIM provisioning; directory-driven entitlements and periodic access reviews.
• Define catalog strategy (e.g., Unity Catalog/Purview/Collibra/Alation) and integrate with CI/CD for automated registration and lineage.
• Design reusable pipeline frameworks with configuration-driven IO, logging, metrics, retry/error handling, and data quality checks.

Skills:

• Data Modeling: Dimensional (star/snowflake), 3NF, Data Vault, business glossary-to-model mapping, SCD types, time-series/event modeling.
• Metadata & Governance: Practical use and integration of Data Catalogs, Lineage,
• Oracle/SQL Server (data modeling, migration/CDC patterns).
• Snowflake (roles, warehouses, performance tuning, tasks/streams, dynamic tables).
• Databricks/Spark (SQL/PySpark, Structured Streaming, Delta Lake; Unity Catalog).
• Security & Compliance: IAM/RBAC, masking, tokenization, encryption; PCI/AML/KYC/GDPR/DPDP awareness.
• Integration & Orchestration: Databricks Workflows, Airflow/ADF, API integrations with catalog tools; schema registry
• Exceptional interpersonal and collaboration skills within a team environment