Ab Initio Lead

Role Overview: 1

Ab Initio lead:

We are seeking an experienced Ab Initio expert to lead and support the migration of legacy Ab Initio workloads to Azure Databricks. The candidate will serve as the onshore technical subject-matter expert, bridging business stakeholders and offshore delivery teams while ensuring data accuracy and delivery quality throughout the migration.

Scope of Engagement

·                Migrate Ab Initio graphs and plans to Azure Databricks ETL jobs, covering assessment, conversion, validation, and help with Production.

·                Translate Ab Initio logic into Databricks (Spark/PySpark/SQL) equivalents.

·                Act as the onshore SME, partnering with business teams and translating requirements into clear specs for offshore delivery.

·                Provide guidance, knowledge transfer, and data debugging support to offshore engineers.

·                Support data validation during migration and help resolve source-side Ab Initio ETL execution issues.

·                Flexible working hours to ensure effective overlap with offshore teams across time zones.

Roles & Responsibilities

Key Responsibilities

·                Analyze and interpret existing Ab Initio graphs, plans, and components to document business and transformation logic.

·                Translate complex Ab Initio functions (transform, rollup, join, partition/departition, lookup, normalize/denormalize, etc.) into target Databricks ETL designs.

·                Partner with business teams to capture requirements and translate them into actionable specifications for offshore development teams.

·                Provide technical guidance and knowledge transfer to offshore engineers executing the conversion.

·                Lead data debugging, reconciliation, and root-cause analysis to resolve discrepancies between source (Ab Initio) and target (Databricks) outputs.

·                Support data validation and reconciliation throughout the migration to ensure source-to-target parity.

·                Troubleshoot and resolve source-side Ab Initio ETL execution issues to keep migration and validation on track.

·                Define reusable conversion patterns, templates, and best practices to accelerate delivery.

·                Support SIT/UAT cycles, validate data parity, and assist with performance tuning on Databricks.

Required Skills & Experience

·                Strong hands-on expertise with Ab Initio (GDE, Co>Operating System, EME, Conduct>It plans, Continuous Flows).

·                Deep understanding of Ab Initio graph design, components, and DML/transform functions.

·                Working knowledge of Azure Databricks, Apache Spark, PySpark, and SQL.

·                Proven experience in legacy ETL modernization or migration programs.

·                Strong data analysis, validation, and debugging skills with the ability to trace logic across complex pipelines.

·                Excellent communication skills to work across business stakeholders and distributed offshore teams.

Preferred Qualifications

·                Experience migrating Ab Initio (or comparable legacy ETL such as Informatica/Teradata) to Databricks or other cloud lakehouse platforms.

·                Familiarity with Azure data services (ADLS, ADF, Azure SQL) and CI/CD for data pipelines.

·                Domain exposure to healthcare, banking, or other regulated, data-intensive industries.

·                Experience leading or mentoring offshore delivery teams.