Job Title: Lead Cloud Data Engineer (Data Mesh & AI Platforms)
Role Overview
We are seeking an experienced and forward-thinking Lead Cloud Data Engineer to drive the development of modern enterprise data platforms and AI-augmented solutions. In this role, you will be critical to supporting large-scale data warehousing operations, architecting centralized and federated data hubs, and spearheading enterprise data mesh modernization programs.
This is a technical leadership role for an engineer who excels at designing distributed cloud architectures and enabling self-serve data capabilities. You will build end-to-end ELT pipelines, implement federated governance, integrate DevSecOps practices, and leverage generative AI frameworks to accelerate the software development lifecycle. If you are passionate about scaling modern data platforms and mentoring engineering teams, we want to hear from you.
Key Responsibilities
Data Engineering & Pipeline Development
ETL/ELT Architecture: Develop and maintain robust end-to-end ingestion, transformation, and consumption-layer pipelines using advanced cloud data services.
Data Quality & Governance: Implement automated data quality frameworks, anomaly detection, data lineage tracking, metadata management, and fine-grained access controls.
Data Mesh Architecture & Self-Serve Platforms
Federated Governance: Design and implement a scalable Data Mesh architecture, establishing domain boundaries, data product specifications, and self-serve infrastructure patterns across modern analytics platforms (e.g., Databricks, Starburst, Collibra, Immuta).
Domain Enablement: Build reusable frameworks, accelerators, and self-serve tools that empower cross-functional domain teams to independently publish, discover, and consume high-quality data products efficiently.
DevSecOps & AI-Augmented Development
Infrastructure Automation: Integrate DevSecOps practices, including CI/CD automation, Infrastructure as Code (Terraform/CloudFormation), security scanning, and disaster recovery planning.
AI Integration: Design and build agentic AI solutions, RAG pipelines, and intelligent automation using modern LLMs and AI-powered development tools to enhance platform capabilities and streamline workflows.
Qualifications & Requirements
Minimum Qualifications
Experience Baseline: 5+ years of specialized experience designing and deploying distributed data architectures and modern cloud platforms.
Cloud & Data Architecture: Deep, hands-on experience building scalable data solutions on major cloud services (e.g., AWS S3, Glue, Lake Formation, EMR, Redshift/Snowflake).
Data Mesh & Governance: Demonstrated implementation of data mesh principles, including domain-oriented data ownership, data-as-a-product, and federated computational governance.
Modern Tooling Proficiency: High familiarity with modern data stacks, including Databricks (Delta Lake, Unity Catalog), federated query engines (Starburst/Trino), and governance tools (Collibra/Immuta).
DevSecOps & IaC: Strong background in CI/CD pipelines, containerization (Docker), and infrastructure automation (Terraform, CloudFormation).
AI & Engineering: Practical experience leveraging generative AI tools for code acceleration, testing, and designing intelligent automation solutions.
Preferred Attributes
Proven track record leading technical design sessions, architecture reviews, and mentoring data engineering teams.
Excellent communication skills, with the ability to bridge complex technical concepts for both technical and executive stakeholders.
Experience navigating hybrid cloud or secure enterprise environments.
Equal Opportunity Employer / Disabled / Protected Veterans
The Know Your Rights poster is available here:
_EEOC_KnowYourRights6.12.pdf
The pay transparency policy is available here:
_%20English_formattedESQA508c.pdf
For temporary assignments lasting 13 weeks or longer, AllSTEM Connections is pleased to offer major medical, dental, vision, 401k and any statutory sick pay where required.
We are committed to working with and providing reasonable accommodations to individuals with disabilities. If you need a reasonable accommodation for any part of the employment process, please contact your staffing representative who will reach out to our HR team.
AllSTEM Connections participates in the E-Verify program in certain locations as required by law. Learn more about the E-Verify program.
_Participation_Poster_ES.pdf
We also consider for employment qualified applicants regardless of criminal histories, consistent with legal requirements, including, if applicable, the City of Los Angeles' Fair Chance Initiative for Hiring Ordinance. Pursuant to applicable state and municipal Fair Chance Laws and Ordinances, we will consider for employment-qualified applicants with arrest and conviction records, including, if applicable, the San Francisco Fair Chance Ordinance. For Los Angeles, CA applicants: Qualified applications with arrest or conviction records will be considered for employment in accordance with the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act.
Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
Additional Skills
(none specified)
AllSTEM Representative Contact Info
Account Executive:
Nichols
Branch Phone:
Location:
Ontario, CA
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: 80184143
- Position Id: d61d0a426b54f94cd70f538e4d45f76b
- Posted 1 day ago
Never miss an opportunity! Create an alert based on the job you applied for.
