Data Architect

Senior Data Architect

FullTIme

Remote (US)

Required Experience : Data Platform, Integration, Azure Databricks & Lakehouse Specialist

Professional Summary

Strong hands-on background in PySpark, Medallion Architecture, Delta Live Tables, Unity Catalog, and MongoDB Atlas integration. Familiar with Databricks Genie for AI-assisted analytics. An effective technical leader who sets architectural direction, mentors teams, and drives delivery. Exposure to Power BI and reporting solutions is a plus, not a core requirement.

Core Skills

• Databricks: Medallion Architecture (Bronze/Silver/Gold), Delta Lake, Delta Live Tables (DLT), Auto Loader, Unity Catalog, Databricks Workflows, Databricks Apps, Databricks Genie
• Languages: PySpark (expert), Python (PEP 8/PEP 20), SQL
• Data Platforms: Azure Databricks, ADLS, Azure Synapse, Snowflake
• Databases & Integration: MongoDB Atlas, SQL Server (JDBC), REST APIs, OAuth (SharePoint), cloudfiles
• Governance: Unity Catalog - metastores, catalogs, schemas, RBAC, lineage, access policies
• DevOps: Git, CI/CD for Databricks workflows and DLT pipelines, JSON/YAML config management
• Reporting (nice-to-have): Power BI, DAX, Power Platform

Key Experience

Medallion Architecture & Databricks Pipelines

• Designed and implemented end-to-end Lakehouse solutions on Azure Databricks across Bronze/Silver/Gold layers with schema enforcement and data quality checks
• Delivered production-grade DLT pipelines and Auto Loader streaming ingestion from ADLS and external sources
• Optimised PySpark jobs for performance and cost - partition tuning, caching, modular function-based code design

Unity Catalog & Governance

• Implemented Unity Catalog as the foundational governance layer - metastores, catalogs, schemas, fine-grained RBAC, lineage, and audit controls
• Standardised metadata management and data discovery across the platform

Data Architecture & Integration

• Architected data flows from MongoDB Atlas, JDBC (SQL Server), REST APIs, and OAuth sources into the enterprise data platform
• Established architectural standards for ingestion, transformation, and consumption layers; led architectural reviews across squads
• Built configuration-driven (JSON/YAML) pipeline frameworks enabling scalable onboarding of new data sources

Databricks Genie

• Familiar with Databricks Genie for enabling natural language querying of data assets and AI-assisted analytics for business users

Leadership

• Mentored engineers on Databricks, PySpark, and Python best practices (PEP 8/PEP 20, naming conventions, modular design)
• Conducted code reviews, enforced coding standards, and drove CI/CD adoption for Databricks workflows and DLT pipelines