Senior / Lead Data Engineer - Hybrid - San Francisco Bay Area, CA - W2 only

Hybrid in San Francisco, CA, US • Posted 1 day ago • Updated 1 hour ago
Contract W2
6 Months
No Travel Required
Hybrid
Depends on Experience
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Data Engineering
  • Azure cloud
  • Azure Data Factory (ADF)
  • Azure Databricks
  • Azure Functions
  • Microsoft Fabric
  • PySpark
  • SQL
  • data warehousing
  • ETL/ELT

Summary

We are seeking a Senior / Lead Data Engineer in San Francisco Bay Area, CA.

 

Job Title: Senior / Lead Data Engineer
Job Type: Contract – 12 Months
Location: San Francisco Bay Area, CA
Work Model: Hybrid or Remote (Travel Required)

Work Arrangement

Candidates can choose either of the following options:

  • Hybrid: 3 days onsite per week in the San Francisco Bay Area, CA.
  • Remote: Work remotely with mandatory travel to the San Francisco Bay Area for 3 consecutive days during the first week of every month.

Job Overview

Client is seeking a highly skilled and experienced Senior / Lead Data Engineer to join its data engineering team for challenging, large-scale, and multi-year data transformation initiatives.

The ideal candidate will bring extensive experience in designing, developing, and implementing scalable data engineering solutions using Azure Native Services, PySpark, SQL, Microsoft Fabric, Azure Data Factory (ADF), and Databricks.

This role offers an excellent opportunity to work on modern cloud-based data platforms, solve complex engineering challenges, and contribute to enterprise-level data transformation projects.

The candidate should possess strong technical expertise, excellent problem-solving skills, and the ability to work independently while collaborating with cross-functional teams.

Key Responsibilities

1. Data Engineering & Pipeline Development

  • Design, develop, and implement scalable data pipelines using PySpark, SQL, Azure, and Databricks.
  • Build and maintain robust ETL/ELT processes for large-scale data processing.
  • Develop efficient data ingestion, transformation, and integration solutions.
  • Optimize data pipelines for performance, scalability, and reliability.

2. Azure Cloud & Microsoft Fabric

  • Leverage Azure Native Services to develop modern cloud-based data solutions.
  • Design and implement data engineering workflows using Azure Data Factory (ADF).
  • Utilize Azure Databricks for large-scale distributed data processing.
  • Work with Microsoft Fabric to build and enhance modern data platforms.
  • Integrate multiple data sources and services across the Azure ecosystem.

3. Data Warehousing & Data Management

  • Design and implement scalable enterprise data warehousing solutions.
  • Apply data warehousing fundamentals and best practices.
  • Develop efficient SQL queries for data transformation, analysis, and reporting.
  • Ensure data quality, consistency, integrity, and reliability.

4. Collaboration & Technical Leadership

  • Collaborate with business stakeholders, architects, and cross-functional engineering teams.
  • Translate business requirements into technical solutions and data architecture.
  • Provide technical guidance and mentorship to junior data engineers.
  • Conduct code reviews and promote best practices in development and implementation.
  • Troubleshoot complex data engineering challenges and production issues.

5. Continuous Improvement

  • Stay updated with emerging technologies and trends in cloud data engineering.
  • Identify opportunities to improve data architecture, pipeline performance, and engineering processes.
  • Contribute to continuous improvement initiatives across the data platform.

Mandatory Skills & Qualifications

  • 10+ years of overall IT experience.
  • 8+ years of hands-on Data Engineering experience.
  • 4+ years of experience working with Azure cloud technologies.
  • Strong hands-on experience with Azure Data Factory (ADF), Azure Databricks, and Azure Functions.
  • 1–2 years of hands-on experience with Microsoft Fabric.
  • Strong programming and development expertise in PySpark.
  • Advanced SQL skills, including querying, manipulating, and analyzing large datasets.
  • Strong understanding of ETL/ELT processes and data warehousing fundamentals.
  • Experience designing and implementing large-scale data pipelines and cloud data solutions.
  • Ability to work independently and deliver in a fast-paced environment.
  • Excellent communication, analytical, and problem-solving skills.

Good-to-Have Skills

  • Databricks Certification.
  • Strong understanding of Data Modeling techniques.
  • Azure Architect Certification.
  • Experience with modern cloud data architecture and enterprise data transformation initiatives.

 

 

Sarum LLC
68-60 Austin Street, Suite 403,
Forest Hills, NY 11375
Phone#

Email :

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 91116024
  • Position Id: 9146644
  • Posted 1 day ago
Contact the job poster
Dilya Zakirova

Dilya Zakirova

Recruiter. @ Sarum LLC

Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.