Informatica to Databricks Migration Engineer

About us:

Creative Information Technology Inc (CITI) is an esteemed IT enterprise renowned for its exceptional customer service and innovation. We serve both government and commercial sectors, offering a range of solutions such as Healthcare IT, Human Services, Identity Credentialing, Cloud Computing, and Big Data Analytics. With clients in the US and abroad, we hold key contract vehicles including GSA IT Schedule 70, NIH CIO-SP3, GSA Alliant, and DHS-Eagle II.

Join us in driving growth and seizing new business opportunities!

Position Description:

WSSC Water currently uses Informatica PowerCenter ETL workflows, mappings, sessions, and related integration processes to support enterprise reporting, analytics, operational data integration, financial reporting, and regulatory reporting. WSSC Water is undertaking a modernization initiative to migrate and transform existing ETL assets from Informatica PowerCenter to Microsoft Azure-based cloud data platforms, including Azure Databricks and Azure Synapse Analytics.

Roles & Responsibilities

·        Assess and inventory existing Informatica PowerCenter workflows, mappings, sessions, schedules, and dependencies.

·        Analyze current ETL architecture and identify migration complexity, risks, and optimization opportunities.

·        Develop migration roadmaps, implementation strategies, and modernization plans.

·        Design Azure-based ETL/ELT solutions using Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage, and Azure Data Factory.

·        Convert and migrate Informatica PowerCenter workflows and mappings to Azure cloud-native solutions.

·        Develop Databricks notebooks, jobs, Synapse pipelines, scripts, and orchestration workflows.

·        Implement ETL best practices including logging, monitoring, error handling, audit tracking, restartability, and parameterization.

·        Optimize ETL performance through partitioning, parallel processing, query tuning, and cloud resource optimization.

·        Integrate Azure data platforms with Oracle, SQL Server, APIs, GIS systems, Power BI, and other enterprise applications.

·        Configure Azure security components including Azure Key Vault, RBAC, Managed Identities, Service Principals, and Microsoft Entra ID.

·        Build and maintain CI/CD pipelines using Azure DevOps and source control repositories.

·        Create technical designs, source-to-target mappings, architecture documentation, and operational runbooks.

·        Develop testing strategies and execute unit, integration, regression, performance, and reconciliation testing.

·        Validate data quality, transformation accuracy, data completeness, and end-to-end processing consistency.

·        Support production deployments, cutover activities, rollback planning, and post-production stabilization.

·        Configure monitoring dashboards, alerts, logging, and operational reporting solutions.

·        Provide technical training, knowledge transfer sessions, and operational support documentation.

·        Collaborate with business stakeholders, architects, developers, QA teams, and production support teams.

·        Provide project status updates, risk assessments, issue tracking, and migration progress reporting.

Required Skills

·        Lead the assessment, design, migration, modernization, testing, and deployment of Informatica PowerCenter ETL workflows to Azure Databricks and Azure Synapse Analytics.

·        Strong experience with Informatica PowerCenter development and migration projects.

·        Hands-on experience with Azure Databricks.

·        Strong expertise in Azure Synapse Analytics.

·        Experience with Azure Data Lake Storage (ADLS).

·        Experience with Azure Data Factory (ADF) or similar orchestration tools.

·        Strong SQL development and query optimization skills.

·        Experience building ETL/ELT pipelines in cloud environments.

·        Experience with data integration involving:

o   Oracle

o   SQL Server

o   APIs

o   File-based integrations

·        Experience implementing CI/CD pipelines using Azure DevOps.

·        Knowledge of source control and release management processes.

·        Experience with monitoring, logging, and operational support frameworks.

·        Understanding of data validation, reconciliation, and data quality processes.

·        Experience with cloud security concepts including:

o   RBAC

o   Azure Key Vault

o   Managed Identities

o   Service Principals

o   Microsoft Entra ID (Azure AD)