Lead Data Engineer - Pipeline Engineering Focus
Client: Mastercard
121 Bloor st E, Toronto, Canada (onsite 5days)
Position Overview
We are looking for a Lead Data Engineer to build, operate, and scale the data pipelines and processing frameworks that power downstream analytics and data products. This role is execution-focused: the primary mandate is engineering excellence - reliable ETL/ELT, strong data quality practices, and operational rigor - rather than direct business stakeholder management. As the subject matter expert on data modeling within the team, you'll also lead engineering activities and mentor other engineers through hands-on technical guidance.
Key Responsibilities
Pipeline Engineering & Operations
- Build and maintain scalable, reliable data pipelines and processing frameworks across the big data ecosystem (Spark, Databricks, Hadoop, PySpark).
- Own the full ETL/ELT lifecycle - ingestion, transformation, aggregation, and processing of large-scale datasets - with a focus on pipeline reliability and automation.
- Design and maintain workflow orchestration using Airflow, ensuring pipelines run reliably and recover gracefully from failure.
- Work with modern table formats (Delta, Iceberg) to support scalable, versioned, and performant data storage.
Data Quality & Governance
- Establish and enforce data quality standards: validation, monitoring, and governance practices across pipelines.
- Ensure secure, compliant data usage in line with data governance and privacy requirements.
- Serve as the team's subject matter expert on data modeling, setting standards for schema design and data structure.
Engineering Leadership
- Lead engineering activities and set technical direction through hands-on expertise, not just process oversight.
- Mentor engineers on pipeline design, performance optimization, and engineering best practices.
- Champion CI/CD, automation, and strong source control practices across the data engineering workflow.
Cross-Functional Collaboration
- Partner with Data Science, Product, Analytics, and Infrastructure/Engineering teams to understand pipeline and data requirements.
- Support analytics and downstream consumers by ensuring pipelines deliver clean, reliable, well-structured data - with limited direct business-facing engagement.
Innovation
- Stay current with and adopt emerging data engineering technologies and practices to continuously improve pipeline architecture and performance.
Required Skills & Experience
- Strong, hands-on experience with Spark, Databricks, Hadoop, PySpark.
- Experience with Airflow or similar orchestration tools for pipeline scheduling and dependency management.
- Experience with modern lakehouse table formats such as Delta Lake and/or Apache Iceberg.
- Proven track record designing and operating pipeline architecture at scale - not just writing individual jobs.
- Strong background in data quality engineering: validation frameworks, monitoring, alerting, and data governance standards.
- Solid grounding in cloud-native data engineering practices.
- Experience with CI/CD, automation, and source control as part of the data engineering SDLC.
- Demonstrated ability to lead through technical expertise and mentor other engineers.
- Strong communication skills - able to clearly explain technical designs and trade-offs to engineering peers.
Not required for this role: GenAI/LLM experience, machine learning, or heavy business-facing analytics work - this role is scoped for pipeline engineering depth over broad business engagement.
Ideal Candidate Profile
A hands-on Lead Data Engineer who is energized by building and operating high-quality, production-grade data pipelines - someone who treats data quality and reliability as first-class engineering problems, and who leads by doing rather than by delegating.
Never miss an opportunity! Create an alert based on the job you applied for.
