Lead Microsoft Fabric Data Engineer

Client

Global asset, investment, and financial products management company operating in the U.S. and 40+ countries.

Position overview

We are looking for an experienced Lead Microsoft Fabric Data Engineer to design and deliver enterprise-scale data and analytics solutions using Microsoft Fabric. This is a deeply hands-on role combining data engineering with architecture ownership. The successful candidate will personally build, optimize, and troubleshoot solutions across OneLake, Lakehouse/Warehouse, Fabric Data Factory, notebooks, SQL/PySpark, and Power BI semantic models, while also defining architectural patterns and engineering standards for the broader team. The role requires someone who can move comfortably between architecture and implementation: making technology and design decisions, validating them through working solutions, resolving production issues, and establishing reusable patterns that other engineers can follow. A key focus is the evolution of an existing enterprise BI and data landscape toward Microsoft Fabric while maintaining and improving the performance, reliability, and scalability of current Power BI and analytical workloads. The estimated salary range for this position is USD 160,000 to 180,000 per year. The role requires three days of office work per week in New York/New Jersey, Newark, or Tampa, Florida.

Responsibilities

• Design and implement end-to-end Microsoft Fabric solutions across OneLake, Lakehouse, Warehouse, Data Factory, notebooks, SQL/PySpark, and Power BI.
• Build and evolve scalable medallion architectures, defining practical patterns for ingestion, transformation, storage, modeling, and consumption.
• Design and implement batch and incremental data pipelines across relational databases, files, APIs, cloud services, and other enterprise data sources.
• Build and optimize enterprise Power BI semantic models, including dimensional models, DAX, aggregations, incremental refresh, and appropriate use of Import, DirectQuery, Direct Lake, and composite models.
• Diagnose and resolve performance issues across data pipelines, Delta tables, SQL workloads, semantic models, reports, and Fabric capacity.
• Define and implement Fabric platform patterns covering workspaces, domains, environments, OneLake data sharing, security, governance, and deployment.
• Apply appropriate architectural and engineering patterns for structured, semi-structured, and large-scale analytical data workloads.
• Implement data quality, reconciliation, lineage, metadata, monitoring, and operational controls.
• Establish and maintain Git-based development, CI/CD, automated testing, deployment, and environment-management practices for Fabric workloads.
• Develop reusable engineering patterns, accelerators, and reference implementations that can be adopted by other teams.
• Support and optimize existing enterprise data and BI solutions while identifying appropriate opportunities for incremental modernization and migration to Fabric.
• Lead technical assessments and proofs of concept through working implementations and measurable outcomes.
• Provide technical leadership across data engineering, BI, architecture, infrastructure, security, governance, and business teams while remaining a direct contributor to implementation and production support.
• Maintain architecture documentation, data flows, technical standards, and implementation guidance based on the solutions and patterns delivered.

Requirements

• 8+ years of experience in data engineering, data architecture, analytics engineering, or enterprise data-platform delivery, with significant hands-on implementation experience.
• Strong hands-on production experience with Microsoft Fabric, with demonstrated implementation experience across several core workloads such as Lakehouse/Warehouse, Data Factory pipelines, notebooks, SQL/PySpark, and Power BI semantic models.
• Deep practical understanding of OneLake, Fabric Lakehouse, Fabric Data Warehouse, Data Factory, Spark/notebook-based engineering, and Power BI.
• Hands-on experience implementing medallion architectures, Delta Lake, Parquet-based storage, and scalable data-processing patterns.
• Strong experience with data ingestion, transformation, orchestration, incremental processing, and large-scale data workloads.
• Strong SQL skills and practical experience with Python and/or PySpark for production data engineering.
• Strong data-modeling fundamentals, including dimensional modeling, star schemas, facts and dimensions, shared/conformed dimensions, and slowly changing dimensions.
• Practical experience building and optimizing enterprise Power BI semantic models, including DAX, Power Query, aggregations, incremental refresh, and performance optimization.
• Strong practical understanding of Direct Lake, Import, DirectQuery, and composite models, including the ability to make storage and connectivity decisions based on data volume, freshness, performance, concurrency, and capacity requirements.
• Experience diagnosing and optimizing performance across SQL, pipelines, data processing, semantic models, Power BI reports, and Fabric workloads.
• Experience working with existing enterprise data and BI environments, not only greenfield implementations, including troubleshooting, optimization, modernization, and incremental migration.
• Experience integrating relational databases, files, APIs, cloud services, and other enterprise data sources into modern analytical platforms.
• Strong practical knowledge of security, access control, governance, lineage, and data protection, including Entra ID, RBAC, RLS/OLS, managed identities, and related enterprise patterns.
• Experience implementing data quality, reconciliation, observability, monitoring, and operational reliability practices.
• Practical understanding of Fabric capacity management and performance, including workload monitoring, capacity constraints, query optimization, and cost/performance considerations.
• Experience with Git-based development, CI/CD, automated testing, and environment management for data workloads.
• Ability to translate complex business and technical requirements into working solutions and take ownership from design through implementation and production support.
• Ability to provide technical leadership and make architecture decisions while remaining a strong hands-on contributor.
• Strong communication skills and ability to collaborate effectively with engineering, architecture, BI, infrastructure, security, governance, and business stakeholders.
• Bachelor's degree in Computer Science, Data Engineering, Information Systems, or equivalent practical experience.
• Availability to visit the client office 2-3 times per week.

Nice to have

• Microsoft Fabric certification (DP-600 and/or DP-700) strongly preferred.
• Relevant Power BI, Azure Data, or architecture certifications.
• Experience working with investment management, asset management, fixed income, financial products, risk, compliance, or distribution data.
• Hands-on experience with Microsoft Purview, Azure Data Lake Storage, Azure Data Factory, Azure Synapse Analytics, Azure Key Vault, and Azure Monitor.
• Experience integrating Microsoft Fabric with SQL Server, Oracle, Snowflake, Databricks, or other enterprise data platforms.
• Experience modernizing or migrating legacy data warehouses, Power BI solutions, Azure Synapse, Azure Data Factory, or other enterprise analytical workloads to Microsoft Fabric.
• Knowledge of data mesh, domain-oriented data architecture, and federated governance applied in enterprise environments.
• Experience with Fabric Real-Time Intelligence, event streaming, or operational analytics.
• Familiarity with TMDL, PBIP, Tabular Editor, XMLA endpoints, or semantic-model automation.
• Exposure to Microsoft AI Foundry, GenAI, agents, or AI-enabled analytics.