Overview
On Site
Depends on Experience
Accepts corp to corp applications
Contract - Independent
Contract - W2
Contract - 12 Month(s)
No Travel Required
Skills
Java
SQL
Best Practices
Problem-Solving
JDBC
Python
Large-Scale
Mentor
Migrations
Data Governance
ETL
Scala
BLOB
Data Services
Networking
Azure Cloud Services
Presentation Skills
VPN
Leadership Skills
Excellent Verbal and Written Communication Skills
Databricks
Azure Data Factory
Scala.
PySpark
Spark SQL
Delta Lake
Job Details
Need Only locals
Job Title: Data Engineer Lead
Location: San Jose, CA (Onsite)
Duration: Long term contract
Visa: Sorry, we can not Sponsor Visa
Visa: Sorry, we can not Sponsor Visa
Educational Qualification*
Bachelor's degree in Computer Science, Information Systems, or a related field.
Job Description of Role* (RNR)
(Mandatory - Minimum 500 words)
(Mandatory - Minimum 500 words)
Overview:
We are looking for a skilled Technical Lead to oversee the design and implementation of large-scale data engineering and migration projects. The ideal candidate will have hands-on expertise in Databricks, Azure Data Factory, and related technologies, with a proven track record of leading at least 3 successful data projects.
Experience Required:
Minimum of 12 years of experience in data engineering and architecture roles.
Must have led 3 or more end-to-end enterprise data projects, Databricks, and Azure technologies.
Must have led 3 or more end-to-end enterprise data projects, Databricks, and Azure technologies.
Primary (Must have skills)*
Primary Skills:
Deep knowledge of architecture and complex data models
Extensive experience with Databricks, PySpark, Spark SQL, and Delta Lake.
Strong command over ETL tools such as Data Services and Azure Data Factory.
Proficient in programming languages: Python, Java, or Scala.
Hands-on with Azure cloud services including networking and security configurations.
Key Responsibilities:
1. Lead the design and implementation of ETL solutions using Azure Data Factory.
2. Guide and mentor team members on data engineering best practices.
3. Develop and maintain complex ETL pipelines using Python, Java, or Scala.
4. Ensure high-quality, secure, and performant data migrations using Azure Blob Storage and Spark JDBC.
5. Drive collaboration across teams to implement advanced analytics and ML models in Databricks.
6. Identify and resolve performance bottlenecks and network-related issues.
7. Ensure adherence to data governance and compliance standards across all data flows.
Strong command over ETL tools such as Data Services and Azure Data Factory.
Proficient in programming languages: Python, Java, or Scala.
Hands-on with Azure cloud services including networking and security configurations.
Key Responsibilities:
1. Lead the design and implementation of ETL solutions using Azure Data Factory.
2. Guide and mentor team members on data engineering best practices.
3. Develop and maintain complex ETL pipelines using Python, Java, or Scala.
4. Ensure high-quality, secure, and performant data migrations using Azure Blob Storage and Spark JDBC.
5. Drive collaboration across teams to implement advanced analytics and ML models in Databricks.
6. Identify and resolve performance bottlenecks and network-related issues.
7. Ensure adherence to data governance and compliance standards across all data flows.
Experience Range: 8+ Years
Secondary Skills (Good To have)*
Secondary Skills:
Understanding of Databricks ML and analytics tools.
Experience resolving networking/VPN issues related to data flow.
Familiarity with data governance, security, and compliance frameworks.
Experience in setting up CI/CD pipelines and using DevOps tools is a plus.
Understanding of Databricks ML and analytics tools.
Experience resolving networking/VPN issues related to data flow.
Familiarity with data governance, security, and compliance frameworks.
Experience in setting up CI/CD pipelines and using DevOps tools is a plus.
Soft skills/other skills (If any)
Analytical Skills: Strong analytical and problem-solving skills, with the ability to interpret complex data sets and derive meaningful insights
Communication Skills: Excellent verbal and written communication skills, with the ability to convey technical information to non-technical stakeholders
Leadership Skills: Proven ability to lead and manage teams, with a track record of successful project delivery.
Presentation Skills: Effectively presenting project updates and outcomes to stakeholders and executives.
Communication Skills: Excellent verbal and written communication skills, with the ability to convey technical information to non-technical stakeholders
Leadership Skills: Proven ability to lead and manage teams, with a track record of successful project delivery.
Presentation Skills: Effectively presenting project updates and outcomes to stakeholders and executives.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.