Job Summary:
We are looking for an experienced Lead Data Engineer to support a large-scale Data Warehouse modernization and migration initiative. The ideal candidate will analyze existing ETL processes built on SSIS, Informatica, SQL Server Stored Procedures, and Oracle PL/SQL packages, understand the underlying business logic, and migrate them to Snowflake SQL and Python-based data processing frameworks.
This role requires strong expertise in Snowflake, Python, AWS, Control-M, GitHub, Jenkins, SSIS, Informatica, Oracle PL/SQL, SQL Server.. In addition to migration activities, the candidate will be responsible for data validation, report adoption support, and decommissioning legacy ETL processes.
Key Responsibilities:
- Analyze and understand existing ETL and database processes built using:
- SSIS
- Informatica
- SQL Server Stored Procedures
- Oracle PL/SQL Packages
- Convert and optimize existing business logic into Snowflake SQL.
- Develop and maintain Python-based data ingestion and orchestration processes.
- Build scalable data pipelines and batch processing solutions.
- Configure, schedule, monitor, and troubleshoot jobs using Control-M.
- Work with AWS services including EC2, S3, AWS Batch, Secrets Manager, and CloudWatch/Log Monitoring.
- Perform Snowflake performance tuning and query optimization.
- Execute data validation and reconciliation between legacy systems and Snowflake.
- Partner with Reporting and Analytics teams to validate reports and ensure successful adoption of migrated data.
- Support UAT, parallel runs, and production cutovers.
- Decommission legacy ETL jobs, scheduling processes, and obsolete data platforms after successful migration.
- Manage source code using GitHub and support CI/CD deployments through Jenkins.
- Collaborate with business and technical teams throughout the migration lifecycle.
Required Skills:
Must Have:
- 10+ years of Data Engineering, ETL, and Data Warehouse experience.
- Strong Snowflake SQL development and performance tuning experience.
- Experience migrating ETL processes from SSIS, Informatica, SQL Server, or Oracle environments.
- Strong Python development skills.
- Hands-on experience with Control-M scheduling and monitoring.
- AWS experience with:
- EC2
- S3
- AWS Batch
- Secrets Manager
- CloudWatch
- GitHub and Jenkins CI/CD experience.
- Experience with data comparison, reconciliation, and migration validation.
Never miss an opportunity! Create an alert based on the job you applied for.