Overview
Skills
Job Details
Azure Data Engineer
MS Fabric SynapseADF SQL ETL PySpark,Digital : Artificial Intelligence(AI),Working as Azure Data Engineer to coordinate with Customer and work on MS Fabric, SynapseADF, SQL,ETL, PySpark
Unified Analytics PlatformWork across Fabric workloads Data Factory (ETL), Synapse Data Engineering (Spark), Synapse Data Warehousing (SQL), and OneLake for storage.
Data Loading ArchitectureDesign data loading patterns, lakehouse architectures, and orchestration processes for enterprise-scale analytics.
Data Preparation Enrichment
Prepare and enrich data for analysis, ensuring data quality and consistency checks.
Implement semantic models for BI and self-service analytics.
Security ComplianceSecure and manage analytics assets, enforce governance policies, and monitor compliance using Microsoft Purview and Managed Identity.
Performance OptimizationMonitor and optimize Fabric pipelines and workloads for cost efficiency and high availability.
Collaboration Stakeholder EngagementWork closely with architects, analysts, and business teams to translate requirements into scalable solutions.
Enable Power BI integration for visualization and reporting.
Data Integration OrchestrationBuild and manage data pipelines using Azure Data Factory (ADF) and Synapse pipelines for batch and incremental ingestion.Automate ETL workflows and implement parameterized pipelines for scalability.
Data Transformation ModelingDevelop PySpark notebooks and SQL scripts to clean, transform, and convert raw files into optimized formats (ParquetDelta).Create views, stored procedures, and dimensional models for reporting and analytics.
Performance OptimizationMonitor Dynamic Management Views (DMVs) in Synapse to analyze system health and reduce bottlenecks.
Implement workload management strategies for query performance and resource governance.
Security GovernanceApply role-based authentication, encryptiondecryption logic, and compliance controls.Automate governance tasks to reduce manual intervention.
Reliability MonitoringBuild failure monitoring systems integrated with Logic Apps for alerting and proactive issue resolution.