Data Engineering Tower Lead

Location: Irvine, CA
Role Type: Enterprise Data Platform & Operations Leadership

Position Summary

We are seeking a highly accomplished Data Engineering Tower Lead to lead our enterprise-wide Data Engineering organization and drive the strategy, delivery, reliability, scalability, and operational excellence of our modern data platform ecosystem.

This leadership role is responsible for overseeing end-to-end data engineering execution, platform operations, production support, modernization initiatives, and engineering governance across multiple teams and programs. The ideal candidate combines deep technical expertise with proven leadership experience managing large-scale data platforms, complex delivery portfolios, and cross-functional stakeholder relationships.

The successful candidate will bring strong expertise in Databricks, Apache Spark, Airflow/Astronomer, Data Engineering, DevOps, Platform Engineering, and Cloud Technologies, along with a track record of building high-performing teams and delivering enterprise-scale data solutions.

Key Responsibilities

Leadership & Delivery Excellence

• Lead and manage the enterprise Data Engineering Tower, including multiple engineering teams, delivery pods, and strategic programs.
• Own end-to-end delivery accountability for data engineering initiatives, ensuring alignment with business priorities and enterprise objectives.
• Drive execution excellence through effective planning, governance, SLA adherence, risk management, and operational oversight.
• Manage program dependencies, escalations, issue resolution, and stakeholder communications.
• Partner closely with business leaders, product owners, architecture teams, and executive stakeholders to deliver business value through data.

Data Engineering & Platform Strategy

• Provide strategic leadership for the design, development, and optimization of scalable, resilient, and high-performance data platforms.
• Establish enterprise standards, best practices, and governance frameworks for data engineering and platform operations.
• Drive platform modernization initiatives, continuous improvement programs, and technology adoption strategies.
• Ensure data quality, reliability, scalability, security, and compliance across the data ecosystem.

Databricks & Apache Spark Leadership

• Serve as the technical leader for Databricks platform strategy, administration, governance, and optimization.
• Drive advanced Apache Spark engineering practices, performance tuning, workload optimization, and cost efficiency initiatives.
• Ensure platform scalability, resource utilization optimization, and operational effectiveness across enterprise workloads.

Workflow Orchestration & Automation

• Define and lead enterprise orchestration strategies leveraging Airflow/Astronomer.
• Establish development standards, governance processes, and reliability practices for workflow management.
• Drive automation initiatives to improve platform efficiency, deployment consistency, and operational productivity.

DevOps, Platform Engineering & Infrastructure Automation

• Champion modern DevOps practices across the data engineering organization.
• Implement and govern CI/CD pipelines, release management processes, and deployment automation frameworks.
• Lead Infrastructure-as-Code (IaC) adoption using technologies such as Terraform, CloudFormation, or equivalent tools.
• Promote platform engineering best practices to enhance developer productivity and operational excellence.

Platform Operations, Reliability & Production Support

• Ensure highly available, secure, scalable, and resilient data platform operations.
• Define and implement enterprise monitoring, observability, alerting, and incident management frameworks.
• Lead production support teams and operational readiness programs to achieve service reliability targets.
• Establish disaster recovery, business continuity, capacity planning, and platform resiliency strategies.
• Drive continuous improvements in platform stability, performance, and operational maturity.

Required Qualifications

• Bachelor''s degree in Computer Science, Information Technology, Engineering, or a related discipline.
• 12+ years of experience in Data Engineering, Big Data, Data Platforms, or related technology domains.
• 5+ years of leadership experience managing enterprise-scale Data Engineering organizations, programs, and teams.
• Strong hands-on and leadership experience with:
  • Databricks
  • Apache Spark
  • Apache Airflow / Astronomer
  • Enterprise Data Engineering architectures
  • ETL/ELT frameworks and large-scale data pipelines
  • CI/CD implementation and DevOps methodologies
  • Infrastructure-as-Code (Terraform, CloudFormation, or similar)
• Proven experience managing enterprise production environments and mission-critical data platforms.
• Strong expertise in platform operations, observability, incident management, reliability engineering, and production support.
• Exceptional communication, executive stakeholder management, team leadership, and organizational influence skills.

Preferred Qualifications

• Experience with cloud platforms including AWS, Azure, and/or Google Cloud Platform.
• Knowledge of Data Governance, Data Quality, Metadata Management, and Data Security frameworks.
• Experience implementing enterprise-scale Data Lakehouse architectures.
• Familiarity with Site Reliability Engineering (SRE) principles and Platform Engineering practices.
• Relevant certifications in Cloud, Databricks, Data Engineering, or Architecture disciplines.
• Experience leading large-scale digital transformation and platform modernization initiatives.

Key Skills

Databricks | Apache Spark | Airflow | Astronomer | Data Engineering | Data Platforms | ETL/ELT | Data Pipelines | Data Lakehouse | DevOps | CI/CD | Infrastructure as Code (IaC) | Platform Engineering | Observability | Monitoring & Alerting | Incident Management | Production Support | SRE | AWS | Azure | Google Cloud Platform | Leadership | Stakeholder Management | Program Delivery