Data Engineer (Microsoft Fabric, Azure Databricks)

Data Engineer Career Opportunity
Our public sector client is seeking a highly skilled and proactive Data Engineer to join their team and support the modernization of their data estate. This role is integral to the migration from legacy systems and the development of scalable, secure, and efficient data solutions using modern technologies, particularly Microsoft Fabric and Azure-based platforms. The successful candidate will contribute to data infrastructure design, data modeling, pipeline development, and visualization delivery to enable data-driven decision-making across the enterprise.

Data Engineer Role and Responsibilities

• Design, build, and maintain scalable ETL/ELT data pipelines using Microsoft Fabric and Azure Databricks.
• Implement medallion architecture (Bronze, Silver, Gold) to support data lifecycle and data quality.
• Support the sunsetting of legacy SQL-based infrastructure and SSRS, ensuring data continuity and stakeholder readiness.
• Create and manage notebooks (e.g., Fabric Notebooks, Databricks) for data transformation using Python, SQL, and Spark.
• Build and deliver curated datasets and analytics models to support Power BI dashboards and reports.
• Develop dimensional and real-time data models for analytics use cases.
• Collaborate with data analysts, stewards, and business stakeholders to deliver fit-for-purpose data assets.
• Apply data governance policies including row-level security, data masking, and classification in line with Microsoft Purview or Unity Catalog.
• Ensure monitoring, logging, and CI/CD automation using Azure DevOps for data workflows.
• Provide support during data migration and cutover events, ensuring minimal disruption.

Data Engineer Required Technical Stack

• Microsoft Fabric
• Azure Databricks
• SQL Server / SQL Managed Instances
• Power BI (including semantic models and datasets)
• SSRS (for legacy support and decommissioning)

Data Engineer Required Skills and Qualifications

• Bachelor's degree in Computer Science, Information Systems, or related field
• 5+ years of experience in data engineering roles, preferably in government or regulated environments
• Proficiency in SQL, Python, Spark
• Hands-on experience with Microsoft Fabric (Dataflows, Pipelines, Notebooks, OneLake)
• Experience with Power BI data modeling and dashboard development
• Familiarity with data governance tools (Microsoft Purview, Unity Catalog)
• Solid understanding of ETL/ELT pipelines, data warehousing concepts, and schema design
• Strong communication and collaboration skills.

Data Engineer Preferred Skills and Qualifications

• Certifications such as Microsoft Certified: Fabric Analytics Engineer or Azure Data Engineer Associate
• Knowledge of CI/CD automation with Azure DevOps
• Familiarity with data security and compliance (e.g., FIPS 199, NIST)
• Experience managing sunset and modernization of legacy reporting systems like SSRS

Data Engineer Required Soft Skills

• Strong analytical thinking and problem-solving abilities
• Ability to collaborate across multidisciplinary teams
• Comfort in fast-paced and evolving technology environments

#RT
#DICEJOBS