Role: Snowflake Migration Lead / Senior Data Engineering Lead
Location: Philadelphia, PA - Hybrid
Hire Type: Full-Time Permanent (No Contract)
Lead Experience in must for the role
Client's is supporting a large US telecom client (EBI Data Engineering) in a time-bound Snowflake migration program — repointing approximately 593 Teradata downstream scripts and 154 Databricks MLOps scripts to Snowflake across domains including NRD, Compensation, Foundational, TechOps, CB, Marketing, LAT, Sales, Finance, Digital, and Products.
We are seeking an experienced Snowflake Migration Lead / Senior Data Engineering Lead to serve as the onshore technical lead for the Client's augmented support team. In this role, you will lead a team of 6–8 Data Engineers (onshore/offshore) through the full repointing lifecycle: SQL syntax conversion (Teradata to Snowflake), Airflow variable and connection updates, script modification, parallel run validation, and production cutover. You will be the primary technical interface between Client's engineers, the client's EBI data team, and the SI Partner who provides SQL conversion utilities and infrastructure.
Repointing Execution & Delivery
● Lead script-level repointing across 593 downstream scripts (PySpark, BTEQ, Shell, SQL, IICS, SSIS) and 154 Databricks MLOps scripts from Teradata/SQL Server/Minio to Snowflake.
● Execute SQL syntax conversion for Teradata BTEQ scripts to Snowflake-compatible SQL, leveraging utility frameworks provided by the SI Partner .
● Drive Airflow variable and connection updates across all migrated workflows, ensuring pipelines point to Snowflake endpoints post-cutover.
● Manage the script modification and change management process, coordinating production-to-parallel sync within the required 1-business-day SLA.
● Own the deployment and sign-off process for each migration wave, maintaining runbooks and go/no-go checklists per domain group.
● Manage data copy requests for Minio-related assets and coordinate initial load and incremental sync activities with the SI Partner.
● Lead and coordinate a team of 6–8 Data Engineer I and Data Engineer II resources (onshore and offshore) across parallel migration workstreams.
● Assign scripts per complexity band (S/M/L) to team members, track daily progress against the LOE estimates, and escalate blockers promptly.
● Review code changes, validate SQL conversion quality, and enforce engineering standards before submission to EBI team for review.
● Mentor junior engineers on Snowflake-specific constructs (virtual warehouses, stages, Snowpark, streams & tasks) and Teradata-to-Snowflake syntax differences.
● Conduct daily stand-ups and provide weekly migration status reports to the Program Manager and client stakeholders.
Validation, Reconciliation & Quality
● Own the parallel run and reconciliation cycle — execute data validation against Teradata source and Snowflake target, targeting sign-off within 2–3 reconciliation iterations per workflow.
● Drive automation of unit testing and validation frameworks in collaboration with the SI Partner, tracking outcomes in the centralized validation dashboard.
● Triage and root-cause data mismatches, performance gaps, and pipeline failures during the parallel run phase; coordinate resolution with SI Partner support as needed.
● Ensure UAT sign-off from the EBI Data team on each domain group before scheduling production cutover.
● Conduct performance and load testing on migrated Snowflake workloads; incorporate SI Partner guidance on query optimization and warehouse sizing.
Stakeholder & SI Partner Coordination
● Serve as the Client's technical point of contact with the EBI client team for the repointing workstream — attending daily syncs, steering calls, and design reviews.
● Coordinate with (SI Partner) on alignment of major SQL conversion utilities prior to team execution, ensuring conversion frameworks are available at program kick-off.
● Translate migration risks and data reconciliation issues into clear business-impact language for non-technical stakeholders.
● Maintain shared documentation: migration decision log, runbooks, issue trackers, and lessons learned repository.
REQUIRED SKILLS & EXPERIENCE
Technical Skills (Must Have)
● 8+ years of data engineering experience; minimum 3 years hands-on with Snowflake (architecture, SQL dialect, performance tuning, RBAC).
● Proven experience migrating Teradata workloads to Snowflake — including BTEQ/ANSI SQL translation, macro conversion, and stored procedure migration.
● Strong PySpark and Python skills; experience working with Databricks is required (MLOps pipelines a strong plus).
● Working knowledge of Apache Airflow — DAG management, variable configuration, connection updates across environments.
● Familiarity with IICS (Informatica Intelligent Cloud Services) and/or SSIS (.dtsx) pipelines in a migration context.
● Solid understanding of data lake and object store patterns (Minio, S3, ADLS) and how they integrate with Snowflake external stages.
● Experience with SQL Server, Teradata, and/or Netezza as source platforms is required.
● Strong Git proficiency; experience with CI/CD pipelines and code change coordination across environments (dev → parallel → prod).
Delivery & Leadership Skills
● Proven experience leading a team of 5+ engineers in a fast-paced, time-bound migration or repointing program.
● Ability to manage scope across complexity bands (S/M/L scripts) and balance team capacity against tight delivery timelines.
● Strong stakeholder management — able to communicate migration status, risks, and escalation paths clearly to both technical and business audiences.
● Experience working in a client-facing consulting or managed services model is strongly preferred.
● Comfortable coordinating with an SI Partner — understanding scope boundaries, handoff points, and utility dependencies without micromanaging.
● Domain experience in US Telecom — CDR processing, subscriber data, network inventory, or revenue assurance datasets aligned to EBI workloads (NRD, Compensation, TechOps, CB, etc.).
● Snowflake SnowPro Core or Advanced Data Engineer certification.
● Prior experience delivering migrations within an Client’s, consulting firm, or analytics-driven managed services engagement.
● Exposure to data observability and reconciliation tooling (Great Expectations, Monte Carlo, or custom validation frameworks).
● Familiarity with Snowflake Snowpark, Streams & Tasks, or Dynamic Tables for replacing legacy Teradata batch patterns.
● Experience with Databricks Unity Catalog or MLOps frameworks in an enterprise telecom context.
A hands-on technical lead who can walk into a live migration program, immediately understand script-level complexity, own execution accountability, and keep a distributed team moving at pace — without needing the SI Partner to hold their hand.
● Someone who can read a Teradata BTEQ script and a Snowflake execution plan in the same breath.
● A communicator who can run a 15-minute daily sync with EBI stakeholders and translate yesterday’s reconciliation failures into a clear resolution path.
● A leader who treats the 593-script scope not as a backlog item but as a delivery commitment with a hard October 2026 deadline.
● A professional who understands the rhythm of parallel run validation — tolerates expected data mismatches, drives iterative reconciliation, and knows when to escalate versus resolve.
Never miss an opportunity! Create an alert based on the job you applied for.
