Key Responsibilities
Design and implement enterprise-grade Databricks Lakehouse architecture using Delta Lake, Unity Catalog, and Databricks SQL, tailored to financial data models
Establish architectural patterns for high-throughput batch and real-time streaming data pipelines using Structured Streaming and Auto Loader
Integrate data pipelines with core banking systems, payment gateways, and third-party data providers
Drive migration strategies from legacy data warehouses and platforms, including Teradata, Oracle, Snowflake, and mainframe environments, to Databricks
Implement enterprise data security, access controls, and fine-grained permissions using Databricks Unity Catalog
Design data masking, tokenization, and encryption strategies for Personally Identifiable Information (PII) and Sensitive Personal Data (SPD)
Ensure data lineage, auditing, and metadata management across multi-cloud and hybrid-cloud deployment environments
Define best practices for PySpark, Delta Live Tables (DLT), and SQL optimization to improve compute efficiency and reduce total cost of ownership
Establish CI/CD, DevOps, and MLOps automation frameworks for code deployment, data quality validation, and model lifecycle management
Implement data quality validation practices using tools such as Great Expectations
Partner with Data Science, Quantitative Analytics, and Business Intelligence teams to support financial modeling, predictive risk analysis, and customer 360 initiatives
Collaborate with business leaders, Chief Risk Officers, and compliance teams to align data platform capabilities with banking business objectives
Provide technical leadership, mentorship, and architecture governance to data engineering and analytics teams
Required Qualifications
5+ years of experience
Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related quantitative field
Master's degree in Computer Science, Data Engineering, Information Systems, or a related quantitative field
Databricks Certified Data Engineer Professional
AWS Certified Solutions Architect
Azure Solutions Architect Expert
Skills
Databricks
Delta Lake
Unity Catalog
Databricks SQL
PySpark
Structured Streaming
Delta Live Tables (DLT)
Cloud Data Architecture
Azure Data Lake Storage
Azure Data Factory
AWS S3
AWS Glue
Google Cloud Platform Cloud Storage and BigQuery
Terraform and Infrastructure as Code
CI/CD Pipelines
MLOps
Data Security and Encryption
Role-Based and Attribute-Based Access Control
Core Banking and Payments Systems Integration
Data Engineering
Technical Leadership
Mentoring and Coaching
Architecture Governance
Data Governance
Data Migration Strategy
Stakeholder Management
Regulatory Compliance
Banking and Financial Services Domain Experience
Data Quality Validation
Schedule
Start date: 2026-11-12
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: compun
- Position Id: MALDC6104842
- Posted 9 hours ago
