Sr. Data Engineer

San Francisco, CA, US • Posted 22 hours ago • Updated 10 hours ago
Full Time
On-site
Company Branding Image
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Analytics
  • FOCUS
  • Workflow
  • Data Governance
  • Big Data
  • Databricks
  • DevOps
  • Data Engineering
  • Continuous Integration
  • Continuous Delivery
  • Apache Hadoop
  • Apache Spark
  • Apache Kafka
  • Python
  • Scala
  • Java
  • SQL
  • NoSQL
  • Database
  • Extract
  • Transform
  • Load
  • Real-time
  • Problem Solving
  • Conflict Resolution
  • Cloud Computing
  • Microsoft Azure
  • Amazon Web Services
  • Google Cloud Platform
  • Google Cloud
  • Apache Flink
  • Streaming
  • Data Processing
  • Data Modeling
  • Distributed Computing
  • Machine Learning (ML)
  • Agile
  • Team Leadership
  • Mentorship
  • Regulatory Compliance
  • Artificial Intelligence

Summary

Join a dynamic data engineering team supporting large-scale analytics and cloud data solutions for a hybrid onsite role in San Francisco, CA. This position focuses on building reliable, high-performance pipelines and modern data platform capabilities using Databricks, Azure Data Factory, and DevOps practices.
Responsibilities
  • Design, build, and maintain scalable data pipelines and analytics solutions for big data environments.
  • Develop and optimize workflows that support efficient, reliable data processing and delivery.
  • Work closely with cross-functional partners to translate data needs into actionable technical solutions.
  • Support data platform operations with a focus on quality, availability, performance, and stability.
  • Apply cloud and DevOps practices to automate data workflows and improve delivery efficiency.
  • Troubleshoot issues across data pipelines and environments to help ensure smooth day-to-day operations.
  • Contribute to data governance, security, and compliance practices within the data ecosystem.
  • Share knowledge and help mentor others while promoting collaborative ways of working.
Skills
  • 8+ years of experience in data engineering with a strong big data background.
  • Hands-on experience with Databricks and Azure Data Factory.
  • Experience with DevOps tools and practices related to data engineering and CI/CD.
  • Strong working knowledge of Hadoop, Spark, Kafka, or similar distributed data technologies.
  • Proficiency in Python, Scala, or Java.
  • Solid understanding of SQL and NoSQL databases.
  • Experience building ETL processes, data pipelines, and real-time or batch data solutions.
  • Strong problem-solving skills and the ability to communicate effectively with technical and business stakeholders.
Preferred Skills
  • Experience with cloud data platforms such as Azure, AWS, or Google Cloud Platform.
  • Knowledge of Flink or other streaming/data processing frameworks.
  • Familiarity with data modeling and distributed computing concepts.
  • Background supporting machine learning pipelines or AI-enabled data solutions.
  • Experience in Agile environments and with team leadership or mentoring.
  • Knowledge of California emissions-related data or compliance work is a plus.

Horizontal Talent is committed to fostering an inclusive workplace where diverse perspectives are valued and everyone has the opportunity to thrive. We welcome candidates of all backgrounds and experiences who are excited to contribute to a collaborative, respectful, and equitable environment.

By applying for this position, you acknowledge and agree that Horizontal Talent may contact you regarding your application using automated technology, including phone calls, SMS/text messages, or email, which may be delivered by our virtual AI recruiter, Alex.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10123528
  • Position Id: 75375
  • Posted 22 hours ago

Company Info

About Horizontal Talent

Horizontal Integration offers two service lines. Our staffing agency places IT, digital and creative talent in both contract and permanent roles. Our digital agency provides digital marketing strategy and technology consulting on a project basis. Many of our clients take advantage of both service lines.

We are a certified Minority Business Enterprise, with more than 200 employees across our offices in Minneapolis, Denver, Dallas and Vadodara, India. We ve been consistently growing and profitable since the company was founded in 2003.

Honors:
Recognized as a "2009 Best Place to Work" by Minneapolis/St.Paul Business Journal.

Fact:
Our people and our organization are deeply rooted in IT and interactive marketing.

Horizontal is proud to be an Equal Opportunity and Affirmative Action Employer. We seek to provide employment opportunities to talented, qualified candidates regardless of race, color, sex/gender including gender identity and/or expression, national origin, religion, sexual orientation, disability, marital status, citizen status, veteran status, or any other protected classification under federal, state or local law.

In addition, Horizontal will provide reasonable accommodations for qualified individuals with disabilities. If you need to request a reasonable accommodation in order to complete the application or interview process, please contact us. All applicants applying must be legally authorized to work in the country of employment.

About_Company_OneAbout_Company_Two
Contact the job poster
SS

Sonu Singh

Recruiter @ Horizontal Talent
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Oakland, California

Today

Full-time

Compensation information provided in the description

Plano, Texas

Today

Full-time

East Bethel, Minnesota

Today

Full-time

USD 41.00 - 78.00 per hour

Houston, Texas

Today

Full-time

Compensation information provided in the description

Search all similar jobs