Hi,
Greetings from Iflowsoft Solutions Inc
Job Title: Sr Microsoft Fabric Data Engineer
Location: Dallas TX (Remote)
Duration: 6+ months
ROLE SUMMARY
Design and build enterprise-scale Microsoft Fabric data platforms with a strong focus on metadata-driven ingestion and orchestration frameworks. Lead the development of reusable, configuration-driven data integration solutions capable of dynamically executing Microsoft Fabric resources including Data Pipelines, Notebooks, Dataflows Gen2, Lakehouses, Warehouses, and Semantic Models. Establish and govern Fabric Medallion Architecture (Bronze/Silver/Gold), implement scalable framework standards, and support migration of enterprise Tableau dashboards from on-prem SQL Server (Epic Clarity ~90%) to governed Microsoft Fabric data products.
KEY RESPONSIBILITIES
• Design and implement metadata-driven ingestion frameworks in Microsoft Fabric to onboard and manage multiple source systems through centralized configurations.
• Build dynamic orchestration frameworks capable of executing Fabric resources including Data Pipelines, Spark Notebooks, Dataflows Gen2, Lakehouses, Warehouses, and Semantic Models based on metadata and business rules.
• Develop reusable and parameter-driven Fabric Data Factory pipelines to support scalable ingestion, transformation, dependency management, auditing, and monitoring.
• Implement framework capabilities for full loads, incremental loads, CDC processing, restartability, error handling, and automated recovery mechanisms.
• Design and maintain execution control tables, metadata repositories, and dependency frameworks to support enterprise-scale orchestration and workload management.
• Stand up the Fabric Medallion Architecture using a hybrid model: Bronze and Silver in the Lakehouse (Delta), Gold in the Fabric Warehouse (T-SQL).
• Build Bronze and Silver layers in the Lakehouse using Spark/PySpark Notebooks and Data Factory Pipelines (source-aligned Bronze; standardized, reusable Silver).
• Build the Gold layer in the Fabric Warehouse as enterprise dimensional models (facts, dimensions, aggregations, KPIs) using T-SQL, loading from Silver via Data Factory pipelines, Spark Notebooks, or Dataflow Gen2.
• Translate parsed Tableau embedded/custom SQL into reusable Silver business logic and Gold fact/dimension data products.
• Build Gold dimensional models and Fabric Semantic Models / Direct Lake datasets that replace embedded SQL and serve as enterprise Tableau published data sources.
• Act as the human-in-the-loop reviewer on automated Bronze/Silver/Gold recommendations by evaluating reusable business concepts, existing Fabric assets, and fact vs dimension vs report-specific mart classifications.
• Implement governance and security including Microsoft Purview cataloging, lineage tracking, sensitivity labels, RLS/OLS, and PHI/HIPAA compliance controls.
• Establish and enforce enterprise standards for metadata management, orchestration, naming conventions, medallion architecture, and governance processes.
• Implement CI/CD using Azure DevOps, Fabric Git Integration, and Deployment Pipelines.
• Lead data reconciliation and validation activities against source systems, partnering with business owners on automated comparisons and sign-off processes.
• Collaborate with Tableau teams to repoint existing dashboards and published data sources to Fabric-based Lakehouse, Warehouse, and Semantic Model assets.
REQUIRED QUALIFICATIONS
• 12+ years of Data Engineering and Data Warehousing experience, including lead-level ownership of enterprise data platform architecture.
• 3+ years of hands-on Microsoft Fabric experience (Lakehouse, Warehouse, Data Factory, Spark Notebooks, Dataflows Gen2, Direct Lake, Semantic Models) or equivalent Azure Synapse/ADF experience with recent Fabric delivery.
• Extensive experience building metadata-driven ingestion and orchestration frameworks for enterprise data platforms.
• Strong experience developing dynamic, configuration-driven ETL/ELT frameworks capable of orchestrating Fabric workloads at scale.
• Proven experience implementing dependency-based job orchestration, execution control frameworks, and operational monitoring solutions.
• Expert T-SQL skills with proven Fabric Warehouse (or Azure SQL DW / Dedicated SQL Pool) delivery of enterprise dimensional Gold models.
• Strong Spark/PySpark expertise for Lakehouse Bronze/Silver transformations and proven Medallion Architecture implementation experience.
• Strong Python development experience for framework automation, metadata processing, orchestration, and platform engineering.
• Strong dimensional modeling (Kimball) expertise for Warehouse-based fact and dimension design.
• Hands-on experience with Microsoft Purview, lineage management, sensitivity labels, and row-level/object-level security implementation.
• Experience implementing CI/CD for data platforms using Azure DevOps, Git-based deployment, and Fabric deployment pipelines.
• Proven track record migrating or modernizing large BI and data warehouse ecosystems onto modern cloud platforms.
• Experience with Agentic AI, MCP-driven automation, and intelligent orchestration concepts.
TECH STACK
• Microsoft Fabric (Lakehouse, Warehouse, Data Factory, Notebooks/Spark, Dataflows Gen2, Direct Lake, Semantic Models)
• OneLake
• Fabric Data Factory
• Metadata-Driven Frameworks
• Dynamic Orchestration Frameworks
• T-SQL
• PySpark
• Python
• Delta Lake
• Azure DevOps
• Git Integration / Deployment Pipelines
• Power BI / Tableau
• SQL Server / Epic Clarity
PREFERRED QUALIFICATIONS
• Microsoft Certified: Fabric Data Engineer Associate (DP-700), Fabric Analytics Engineer Associate, and/or Azure Data Engineer Associate (DP-203).
• Demonstrated experience designing enterprise metadata-driven ingestion and orchestration frameworks in Microsoft Fabric.
• Experience building centralized execution engines, dependency-management frameworks, and operational monitoring solutions for Fabric workloads.
• Epic Clarity / Caboodle experience.
• Experience integrating Fabric Warehouse (SQL Endpoint, XMLA Endpoint, Direct Lake) as published data sources for Tableau consumption.
• Familiarity with SQL parsing, normalization, and automation techniques using SQLGlot and Python.
• Experience leveraging AI-assisted development, Agentic AI, MCP Servers, and automation frameworks for accelerating Fabric platform delivery and operations.
• Experience supporting large-scale healthcare, HIPAA, or regulated data environments.
Iflowsoft is an Equal Opportunity Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, disability, military status, national origin or any other characteristic protected under federal, state, or applicable local law