W2 _ Lead Data Engineer _ San Francisco, CA - Hybrid

Position: Lead Data Engineer

Location: San Francisco, CA - Hybrid

Duration: 9+Months with possible extension

Interviews: (4 Rounds): All the interviews within one week.

1. General Interview

2. Technical Interview

3. Project Interview

4. Client Interview

Main skills: SQL, Databricks, ADF, DataStage (or other ETL tool), SSAS cubes, Cognos/other, Tableau/other, ThoughtSpot

Job Description:

We are seeking a Lead Data Engineer to design, implement, and maintain our data infrastructure and business intelligence solutions. This role requires a technical expert who can architect scalable data solutions while collaborating across teams to drive business insights.

Required Experience:

8-10 years of software development and deployment experience

At least 7 years of hands-on experience with:

SQL

Databricks

Azure Data Factory (ADF)

DataStage (or other ETL tools)

SSAS cubes

BI tools (Cognos, Tableau, ThoughtSpot, etc.)

Strong Database knowledge (SQL & No-SQL preferred)

Advanced SQL and Python skills

Experience working with APIs for data collection/ingestion

Key Responsibilities:

Technical Development:

Write SQL for processing raw data, Kafka ingestions, ADF pipelines, data validation, and QA

Lead, design and implement innovative analytical solutions using Tableau, SQL, and Big Data technologies

Experience with storing, joining, filtering, and analyzing data using SQL, Spark, Hive, etc.

Data Architecture & Design:

Design comprehensive data architecture and data modeling strategies for BI systems

Create logical and physical data models using best practices to ensure high data quality and reduced redundancy

Collaborate with stakeholders to translate business requirements into robust, scalable data models

Perform detailed analysis of business problems and technical environments to design quality solutions

Integration & Implementation:

Integrate data from various sources (databases, warehouses, external systems).

Develop strategies for ETL processes ensuring data accuracy, consistency, and availability.

Evaluate and select appropriate BI tools based on organizational needs.

Run POCs and implement BI software, including visualization tools and analytics solutions.

Optimization & Governance:

Monitor and optimize BI system performance for efficient data retrieval, processing, and reporting

Identify and resolve performance bottlenecks through query tuning and data structure optimization

Establish data governance policies ensuring data quality, consistency, and security

Define access controls, privacy measures, and retention policies to meet regulatory requirements

Collaboration & Leadership:

Build partnerships with architects, technical leads, and stakeholders across functional groups.

Actively participate in code reviews and testing to ensure adherence to best practices.

Work with product and engineering teams to evaluate requirements and new features.

Build and foster a high-performance engineering culture.

Mentor team members and provide tools and motivation for success.

Additional Skills & Requirements:

Experience building data ingestion pipelines and data warehouse architecture

Ability to write design documentation, source-to-target mapping documentation, and manage Confluence pages

Experience converting business functionalities into technical Jira stories

Strong experience with data modeling, design patterns, and building scalable BI solutions

Knowledge of cloud platforms (Azure, AWS or equivalent)

Experience analyzing large datasets to identify trends, patterns, and outliers

Regards,

Bala Krishna

Talent Acquisition Specialist

NasTech Global Inc,

Ph#:

Email:

CONFIDENTIAL AND PROPRIETARY. This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute, or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing, or taking any action in reliance on the contents of this information is strictly prohibited. NasTech Global Inc., All rights reserve.