Microsoft Fabric Data Integration Engineer

TECHNOGEN, Inc. is a Proven Leader in providing full IT Services, Software Development and Solutions for 15 years.

TECHNOGEN is a Small & Woman Owned Minority Business with GSA Advantage Certification. We have offices in VA; MD & Offshore development centers in India. We have successfully executed 100+ projects for clients ranging from small business and non-profits to Fortune 50 companies and federal, state and local agencies.

Microsoft Fabric Data Integration Engineer

Duties and Responsibilities:

• Microsoft Fabric Solution Design and Development
  • Design and implement a scalable data integration solution using Microsoft Fabric.
  • Configure and utilize Microsoft Fabric Data Factory, Data Pipelines, Dataflows Gen2, OneLake, Lakehouse, and Warehouse components, as appropriate.
  • Develop automated ETL/ELT processes to import, transform, and consolidate enterprise data.
  • Establish appropriate staging, transformation, and integrated data storage structures.
  • Configure data refresh schedules, processing logs, error handling, and automated notifications.
  • Optimize data pipeline performance and ensure efficient resource utilization.
• SAP and Workday Data Extraction and Integration
  • Analyze SAP and Workday data structures and available integration interfaces.
  • Design and develop secure, automated processes to extract required data elements from both systems.
  • Utilize appropriate supported connectors, APIs, web services, or approved extraction methods.
  • Implement initial data loads and recurring incremental or full data refresh processes, as appropriate.
  • Coordinate with SAP and Workday system administrators and functional subject matter experts to establish data access and validate source information.
  • Ensure source-system data integrity and compliance with applicable security requirements.
• Data Matching, Joining, and Reconciliation
  • A critical responsibility of this position is the ability to accurately associate and consolidate records from SAP and Workday.
  • The contractor will:
  • Identify, analyze, and validate the designated common key data element used to match corresponding records between SAP and Workday.
  • Develop SQL-based joins, lookup operations, and transformation logic to associate records accurately across both systems.
  • Establish appropriate data standardization rules to address differences in data types, field formats, and naming conventions.
  • Identify and resolve duplicate records, missing identifiers, inconsistent values, and unmatched records.
  • Develop and maintain cross-reference or mapping tables when source-system identifiers are not directly compatible.
  • Implement automated reconciliation procedures to validate the completeness and accuracy of the integrated data.
  • Develop exception reports to identify records that cannot be successfully matched or reconciled.
  • Work with designated DLS business and technical personnel to validate matching rules and resolve data discrepancies.

The successful candidate must demonstrate experience joining and reconciling data from disparate enterprise systems using a common identifier, including the ability to address data quality and matching exceptions.

• Data Quality, Security, and Governance
  • Develop and implement automated data quality and validation procedures.
  • Verify record counts, key field values, and other appropriate reconciliation measures against source-system information.
  • Establish controls to identify incomplete, inaccurate, or inconsistent data.
  • Implement role-based access controls and appropriate security measures to protect sensitive financial and personnel information.
  • Maintain appropriate audit trails, processing logs, and data lineage.
  • Ensure the solution complies with applicable MGA information security, confidentiality, and data governance requirements.
• Testing, Documentation, and Knowledge Transfer
  • Develop and execute unit, integration, and data reconciliation tests.
  • Support user acceptance testing with designated DLS personnel.
  • Document the solution architecture, source-to-target mappings, transformation rules, data matching logic, and pipeline configurations.
  • Prepare operational procedures covering scheduled processing, monitoring, troubleshooting, and recovery.
  • Provide technical knowledge transfer to designated DLS information technology personnel.
  • Recommend improvements to support long-term maintenance, reliability, and future expansion of the solution.
    
    Education and Years of Experience:
• Bachelor's degree in Computer Science, Information Systems, Information Technology, Data Engineering, or a related technical field, or an equivalent combination of education and relevant professional experience.
• Minimum of five years of professional experience in data engineering, data integration, database development, or a related discipline.
• Demonstrated hands-on experience developing and deploying solutions using Microsoft Fabric.
• Experience designing, developing, and maintaining automated ETL/ELT data pipelines.
• Experience integrating information from enterprise applications, including ERP and/or HRIS systems.
• Strong proficiency in SQL, including complex joins, transformations, data validation, and reconciliation.
• Demonstrated experience matching and consolidating records from multiple systems using common identifiers.

Required Skills & Certifications:

• Microsoft Fabric
  • Data Factory and Data Pipelines
  • Dataflows Gen2
  • OneLake and Lakehouse
  • Fabric Warehouse
  • Data transformation and orchestration
  • Pipeline monitoring and troubleshooting
• Data Integration and Development
  • Advanced SQL development
  • ETL/ELT architecture and implementation
  • Data modeling and source-to-target mapping
  • Data matching, reconciliation, and exception handling
  • REST APIs, JSON, and web services
  • Secure enterprise data integration
• Enterprise Systems
  • Experience extracting data from SAP and/or Workday and writing/creating reports per user requirements once Fabric is accurately populated.
  • Understanding of enterprise financial and human resources data structures
  • Experience developing interfaces between disparate enterprise applications
  • Understanding of data security and access management requirements

Additional Requirements:

• Direct, hands-on experience integrating data from both SAP and Workday.
• Experience with SAP financial data and Workday human resources or payroll data.
• Experience with SAP ECC 6.0 EHP 6, SAP data extraction interfaces, or SAP integration technologies.
• Experience with Workday Reports-as-a-Service (RaaS), Workday Web Services, or Workday integration tools.
• Proficiency in Python, PySpark, or Power Query.
• Experience with Microsoft Azure Data Factory and related Azure data services.
• Experience developing datasets for Microsoft Power BI.
• Experience implementing automated financial and personnel data reconciliation solutions.
• Microsoft Fabric or relevant Microsoft Azure certifications.
• Previous experience supporting government agencies or organizations handling sensitive financial and personnel information.
• Experience delivering documented, production-ready data integration solutions within defined contractual timeframes.
• Performance Expectations - The contractor will be expected to:
  • Independently manage assigned technical activities and meet agreed-upon project milestones.
  • Provide regular status updates to designated DLS project management and technical personnel.
  • Identify and communicate project risks, technical dependencies, and data quality issues promptly.
  • Deliver a reliable, secure, and maintainable integration solution.
  • Demonstrate that SAP and Workday records are accurately joined using the approved matching criteria.
  • Ensure that identified discrepancies are appropriately documented and resolved or reported.