Role: Senior Databricks Data Engineer
Location: Calgary, Canada
Rate: CAD /hr
Real-Time & Batch Integration: Configure and manage continuous data ingestion pipelines using Apache Kafka event streams for real-time market data, alongside scheduled batch processing of Parquet files and Delta Lake tables.
Medallion Data Modeling: Architect and implement Databricks Lakehouse data models (Bronze/Silver/Gold layers) tailored for energy commodities, ensuring high performance, data lineage, scalability, and compute efficiency.
Cross-Desk Reporting Delivery: Collaborate with Front, Middle, and Back Office stakeholders to transform raw transactional data into curated datasets supporting critical reporting for risk (VaR), PnL, scheduling, and regulatory compliance.
Databricks Platform Architecture: Serve as a Subject Matter Expert in designing, configuring, and securing Databricks environments, including Unity Catalog governance, cluster optimization, CI/CD deployment workflows, and platform performance tuning.
Data Engineering & Optimization: Build scalable ETL/ELT workflows using PySpark and SQL, optimizing large-scale distributed data processing across cloud-native lakehouse architectures.
Governance & Reliability: Implement monitoring, alerting, and data quality frameworks to ensure reliable, auditable, and compliant enterprise data pipelines.
Never miss an opportunity! Create an alert based on the job you applied for.