Data Engineer

San Francisco, CA, US • Posted 22 hours ago • Updated 10 hours ago
Full Time
On-site
Company Branding Image
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Analytics
  • Sourcing
  • Reporting
  • Technical Drafting
  • Scalability
  • Testing
  • Issue Resolution
  • Continuous Improvement
  • Documentation
  • Agile
  • Sprint
  • Estimating
  • Mentorship
  • IT Management
  • Google Cloud Platform
  • Google Cloud
  • SQL
  • Extract
  • Transform
  • Load
  • ELT
  • Data Modeling
  • Warehouse
  • Cloud Computing
  • Git
  • Continuous Integration
  • Continuous Delivery
  • Analytical Skill
  • Conflict Resolution
  • Problem Solving
  • Communication
  • Collaboration
  • Apache Airflow
  • Workflow
  • Orchestration
  • Apache Kafka
  • Streaming
  • PySpark
  • Data Processing
  • Python
  • Data Engineering
  • Data Quality
  • Meta-data Management
  • Data Governance
  • Retail
  • Supply Chain Management
  • Logistics
  • Warehouse Management
  • Distribution
  • Artificial Intelligence

Summary

Join a dynamic Supply Chain Data & AI team as a Lead Data Engineer, where you'll help design and deliver scalable cloud-based data solutions that power analytics and AI-driven decisions. This onsite role offers the opportunity to work across Sourcing, Transportation, and Warehouse Management domains while partnering with cross-functional teams to build high-quality, enterprise data products.
Responsibilities
  • Design and build scalable data pipelines and data products on Google Cloud Platform.
  • Develop and optimize enterprise data solutions using Dataproc, BigQuery, SQL, and dbt.
  • Create reliable data models that support both analytical and operational reporting needs.
  • Build efficient ETL/ELT processes to ingest, transform, and publish data from multiple enterprise systems.
  • Partner with product managers, solution architects, data architects, analysts, and business stakeholders to turn requirements into technical solutions.
  • Lead technical design discussions and participate in code reviews to maintain engineering quality and consistency.
  • Improve performance, reliability, scalability, and cost efficiency across cloud data platforms.
  • Support monitoring, testing, production issue resolution, and continuous improvement efforts.
  • Contribute to reusable frameworks, documentation, and engineering standards.
  • Work within Agile delivery teams and help with sprint planning, estimation, and backlog refinement.
  • Mentor team members and help strengthen technical practices across the team.
Skills
  • 8+ years of experience in data engineering with technical leadership on enterprise projects.
  • Strong hands-on experience with Google Cloud Platform.
  • Advanced proficiency in Dataproc, BigQuery, SQL, and dbt.
  • Solid understanding of modern ETL/ELT architecture and large-scale data processing.
  • Experience with data modeling, including dimensional, normalized, and warehouse design approaches.
  • Background building scalable, maintainable cloud-native data pipelines.
  • Experience with Git, CI/CD pipelines, and engineering best practices.
  • Strong troubleshooting, analytical, and problem-solving abilities.
  • Clear written and verbal communication skills with the ability to collaborate across teams.
Preferred Skills
  • Experience with Apache Airflow for workflow orchestration.
  • Exposure to Apache Kafka or other streaming technologies.
  • Working knowledge of PySpark for distributed data processing.
  • Proficiency in Python for automation, utilities, and data engineering tasks.
  • Familiarity with data quality, metadata management, and data governance practices.
  • Experience in retail, supply chain, transportation and logistics, warehouse management, or distribution center operations.
  • Self-driven mindset with a strong sense of ownership and commitment to high-quality delivery.
  • Ability to balance technical excellence with business priorities in a fast-paced environment.

Horizontal is committed to fostering an inclusive, equitable, and welcoming environment where diverse perspectives are valued and everyone has the opportunity to contribute and grow. We encourage candidates from all backgrounds to apply and bring their unique strengths to the team.

By applying for this position, you acknowledge and agree that Horizontal Talent may contact you regarding your application using automated technology, including phone calls, SMS/text messages, or email, which may be delivered by our virtual AI recruiter, Alex.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10123528
  • Position Id: 75481
  • Posted 22 hours ago

Company Info

About Horizontal Talent

Horizontal Integration offers two service lines. Our staffing agency places IT, digital and creative talent in both contract and permanent roles. Our digital agency provides digital marketing strategy and technology consulting on a project basis. Many of our clients take advantage of both service lines.

We are a certified Minority Business Enterprise, with more than 200 employees across our offices in Minneapolis, Denver, Dallas and Vadodara, India. We ve been consistently growing and profitable since the company was founded in 2003.

Honors:
Recognized as a "2009 Best Place to Work" by Minneapolis/St.Paul Business Journal.

Fact:
Our people and our organization are deeply rooted in IT and interactive marketing.

Horizontal is proud to be an Equal Opportunity and Affirmative Action Employer. We seek to provide employment opportunities to talented, qualified candidates regardless of race, color, sex/gender including gender identity and/or expression, national origin, religion, sexual orientation, disability, marital status, citizen status, veteran status, or any other protected classification under federal, state or local law.

In addition, Horizontal will provide reasonable accommodations for qualified individuals with disabilities. If you need to request a reasonable accommodation in order to complete the application or interview process, please contact us. All applicants applying must be legally authorized to work in the country of employment.

About_Company_OneAbout_Company_Two
Contact the job poster
KA

KP Aishwarya

Recruiter @ Horizontal Talent
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

San Francisco, California

Today

Full-time

San Francisco, California

Today

Full-time

San Francisco, California

Today

Full-time

Compensation information provided in the description

Coppell, Texas

Today

Full-time

Search all similar jobs