Microsoft Fabric Data Engineer

Job Title: Microsoft Fabric Data Engineer
Location: Phoenix, AZ or Chicago, IL or Remote

Design and operate endtoend data pipelines on Microsoft Fabric, from ingestion into OneLake to curated Delta Lake outputs and interactive reporting.

Responsibilities

• Build and maintain ELT/ETL pipelines using Microsoft Fabric Data Factory (data pipelines, Dataflow Gen2) to ingest data from multiple sources into OneLake.
• Apply transformations to cleanse, standardize, and enrich data so it is ready for analytics and reporting.
• Handle large datasets in Lakehouse and Warehouse, optimizing storage, partitioning, and retrieval for performance and cost.
• Implement automation for data processing and integration workflows (scheduling, triggers, dependencies) to reduce manual intervention.
• Work closely with Platform Architects to ensure Fabric infrastructure (workspaces, capacities, Lakehouses, Warehouses) supports current and future data requirements.
• Partner with report developers / BI teams to ensure data structures, semantics, and performance support selfservice analytics and dashboarding.
• Ensure code reusability and parameterization (reusable pipelines, templates, configurationdriven designs).
• Build interactive, intuitive reports and dashboards using Microsoft Fabric reporting capabilities (Power BI in Fabric).

Qualifications & Skills

Core Fabric & Data Engineering

• Data Factory in Fabric
• Designing and orchestrating data ingestion and transformation pipelines (ETL/ELT).
• Using Data pipelines and Dataflow Gen2 for copy, transformation, and orchestration.
• Sparkbased Data Engineering
• Handson experience with Notebooks (PySpark, Spark SQL, Scala) in Fabric.
• Using Spark Job Definitions for complex data processing, cleansing, enrichment, and largescale transformations directly on OneLake data.
• Lakehouse & Storage
• Creating and managing Lakehouse items (Delta tables, files) as the primary landing, staging, and curated zones.
• Strong understanding of OneLake / ADLS Gen2 structures, Delta Lake format, partitioning strategies, and use of Shortcuts where applicable.
• Monitoring & Optimization
• Using Monitoring Hub to track pipeline runs, Spark job performance, failures, and SLAs.
• Tuning Spark jobs and data layouts for performance, reliability, and cost optimization.
• Data Governance & Security
• Implementing data cleansing and quality rules.
• Transforming raw data into curated Delta tables within Lakehouses or Warehouses.
• Applying data security, access control, and governance principles to pipelines, Lakehouses, and other Fabric items.

Soft Skills & Background

• Excellent communication and collaboration skills, able to work with architects, engineers, and business/BI teams.
• Bachelor s degree in Computer Science, Engineering, or a related field.
• Cloud certifications (Azure and/or AWS) are a plus.
• Manufacturing domain experience is a strong plus (shopfloor data, MES, ERP, supply chain).
• Selfdriven, able to own initiatives endtoend and drive projects to successful delivery with minimal supervision.

Keywords: Microsoft Fabric, Fabric Data Factory, Data Factory in Fabric, Dataflow Gen2, data pipelines, ETL, ELT, OneLake, Lakehouse, Delta Lake, Delta tables, ADLS Gen2, Shortcuts, Spark, PySpark, Spark SQL, Scala, Spark Job Definition, Fabric Notebooks, data ingestion, data transformation, data cleansing, data enrichment, data modeling, data partitioning, data optimization, Monitoring Hub, pipeline monitoring, Spark performance tuning, data quality rules, curated Delta tables, data warehouse, Fabric Warehouse, Power BI, Fabric reporting, interactive dashboards, reusable pipelines, parameterization, automation, scheduling, triggers, dependencies, data security, data governance, access control, Azure, Azure certifications, AWS, cloud certifications, manufacturing domain, MES, ERP, supply chain analytics, Platform Architect collaboration, BI developer partnership, selfdriven, project delivery

About VDart Group
VDart Group is a global leader in technology, product, and talent solutions, serving Fortune 500 clients in 13 countries. With over 4,000 professionals worldwide, we deliver innovation, operational excellence, and measurable outcomes across industries. Guided by our commitment to People, Purpose, and Planet, VDart is recognized with an EcoVadis Bronze Medal and as a UN Global Compact member, reflecting our dedication to sustainable practices.