Snowflake DBT Lead

Key Responsibilities

Design develop and maintain robust ETL/ELT pipelines using SQL DBT and Snowflake

Perform data matching processes using various methodologies Union Union Precision Columns Joined Precision Columns

Configure data loads and creating data pipeline workflows for customer

Write optimize the DMLDDL Operations in snowflake

Enabling and validating match process to Build at client level

Match process automation Snowflake data ingestion

Converting multi cloud social data into structured data in Snowflake

Onboard data files and segments into Snowflake tables

Migrate data processes eg from EMR to Snowflake

Investigate data issues including low match rates unusual record counts spikes truncated data and discrepancies after adding fields

Convert Snowflake views to CSV files for external use analysis

Manage Snowflake roles and grant access to views

Modify data consumption procedures

Research and implement new data onboarding processing workflows

Generate synthetic IDs and map them to client data eg for partner matching like Yahoo

Split and compress data files eg using bzip

Use vendor specific APIs for data upload and download eg Yahoo API

Perform impact analysis on backfill process

Own the technical roadmap including performance tuning cost optimization and platform scalability

Build and optimize modular reusable dbt models for analytics and reporting

Implement and enforce data governance quality checks and documentation standards

Guide and mentor a team of developers and analysts ensuring adherence to best practices

Collaborate with data analysts product teams and business stakeholders to understand data requirements

Implement data quality checks testing frameworks and documentation standards

Monitor and troubleshoot data workflows ensuring reliability and performance

Participate in code reviews CICD processes and version control using Git

Contribute to the architecture and design of the data platform and analytics ecosystem

Monitor and troubleshoot production data pipelines and ensure SLAs are met

Required Skills Qualifications

5-10 years of experience in data engineering ETL development or analytics engineering

Expert level proficiency in SQL and Snowflake

Snowflake Databases Schemas Tables Views Procedures Datashare Roles Staging Production Work area Sandbox Partition Date SQL SELECT WHERE ORDER BY qualify row number MAX etc

Handson experience with DBT Data Build Tool in enterprise environments

Experience with Airflow Jobs Orchestration Automation Orchestration Dag Daily Weekly Cadence scripting

Experience in Python scripting

File Formats Compression Parquet CSV GZ bzip

Exposure and Experience in matching methodologies Union Union Precision Columns Joined Precision Columns

Strong understanding of data architecture data modelling and pipeline orchestration

Strong proficiency in SQL for data transformation analysis and performance tuning

Handson experience with DBT in production environments

Experience in performing DMLDDL operations in snowflake

Data Processing Techniques Deduplication list Filtering Data Merging Data Copying Backfilling

API Interaction Fetching data via API using Yahoo API for partner match upload download

Deep understanding of Snowflake architecture including virtual warehouses data sharing and performance optimization

Experience with data modelling star snowflake schema data governance and metadata management

Familiarity with cloud platforms AWS Azure or Google Cloud Platform Esp AWS Cloud Storage s3

Proficiency in Git Bit bucket CICD pipelines and Agile methodologies

Excellent problem-solving communication and stakeholder management skills

Excellent leadership communication and stakeholder management skills"