Job Title: Data Engineer LeadLocation: San Jose, CA (Onsite)

Job Title: Data Engineer Lead

Location: San Jose, CA (Onsite)

Duration: Long term contract

Job Description of Overview: We are looking for a skilled Technical Lead to oversee the design and implementation of large-scale data engineering and migration projects. The ideal candidate will have hands-on expertise in Databricks, Azure Data Factory, and related technologies, with a proven track record of leading at least 3 successful data projects.

Experience Required:

Minimum of 12 years of experience in data engineering and architecture roles.

Must have led 3 or more end-to-end enterprise data projects, Databricks, and Azure technologies.

Primary (Must have skills)*

Primary Skills:

Deep knowledge of architecture and complex data models

Extensive experience with Databricks, PySpark, Spark SQL, and Delta Lake.

Strong command over ETL tools such as Data Services and Azure Data Factory.

Proficient in programming languages: Python, Java, or Scala.

Hands-on with Azure cloud services including networking and security configurations.

Key Responsibilities:

1. Lead the design and implementation of ETL solutions using Azure Data Factory.

2. Guide and mentor team members on data engineering best practices.

3. Develop and maintain complex ETL pipelines using Python, Java, or Scala.

4. Ensure high-quality, secure, and performant data migrations using Azure Blob Storage and Spark JDBC.

5. Drive collaboration across teams to implement advanced analytics and ML models in Databricks.

6. Identify and resolve performance bottlenecks and network-related issues.

7. Ensure adherence to data governance and compliance standards across all data flows.

Expierence Range

10+ Years

Secondary Skills (Good To have)*

Secondary Skills:

Understanding of Databricks ML and analytics tools.

Experience resolving networking/VPN issues related to data flow.

Familiarity with data governance, security, and compliance frameworks.

Experience in setting up CI/CD pipelines and using DevOps tools is a plus.

Soft skills/other skills (If any)

Analytical Skills: Strong analytical and problem-solving skills, with the ability to interpret complex data sets and derive meaningful insights

Communication Skills: Excellent verbal and written communication skills, with the ability to convey technical information to non-technical stakeholders

Leadership Skills: Proven ability to lead and manage teams, with a track record of successful project delivery.

Presentation Skills: Effectively presenting project updates and outcomes to stakeholders and executives.

Educational Qualification*

Bachelor s degree in Computer Science, Information Systems, or a related field.