Lead Data Engineer to San Francisco, CA (Hybrid) - Local to CA Only

ROLE: DATA ENGINEER LEAD (EDW to DATABRICKS MIGRATION)

LOCATION: SAN FRANCISCO, CA (HYBRID)

Final Interview will be F2F in client office.

JOB OVERVIEW:

• Seeking a highly experienced Senior Data Engineer to support a large-scale enterprise data warehouse modernization initiative. The role focuses on migrating a 20-year-old SQL-based EDW environment to Databricks, involving deep analysis of legacy ETL pipelines, data models, and reporting dependencies. This is not a lift-and-shift migration the work requires full re-engineering of ETL pipelines, redesigning data models, and modernizing reporting integrations.
• The ideal candidate is both architecturally strong and hands-on, with extensive experience in SQL, Datastage, Python, and Databricks, and has previously worked on complex data platform migration projects.

JOB DESCRIPTION:

• We are looking at 8-10 years of software development and deployment experience with at least 7 years of hands-on experience with SQL, Databricks, ADF, DataStage (or other ETL tool).
• Write SQL for processing raw data, Kafka ingestions, ADF pipelines, data validation, and QA.
• Knowledge working with APIs to collect or ingest data.
• Experience with advanced SQL and Python.
• Strong Database knowledge, SQL & No-SQL preferred.