Data Engineer (Azure Data Platform)

Job Description

Must Have Technical/Functional Skills:

Cosmos; PL/SQL; MSBI; Digital : Microsoft Power BI; Digital : Big Data Platforms

Roles & Responsibilities:

• Automated Incident Detection & Remediation
• Design and implement automated incident detection and remediation frameworks for data platforms and pipelines.
• Shift incident response from reactive manual processes to proactive, policy-driven workflows.
• Improve mean time to detect (MTTD) and mean time to remediate (MTTR) through automation.
• DataOps Automation & AIOps Frameworks
• Build and operationalize DataOps automation frameworks, incorporating AIOps where applicable for anomaly detection and predictive alerting.
• Reduce operational noise and repetitive manual tasks through intelligent automation.
• Establish standardized operational runbooks and automated recovery patterns.
• Platform Observability & Telemetry Standardization
• Define and implement platform-wide observability and telemetry standards for data workloads.
• Enable consistent metrics, logging, alerting, and health signals across platforms and pipelines.
• Ensure operational visibility is actionable and supports automated response.
• Self-Service Operational Tooling
• Build self-service operational tooling that enables engineering teams to debug, monitor, and resolve issues independently.
• Reduce operational dependencies on centralized teams and vendors.
• Empower teams with standardized dashboards, insights, and tooling.
• Automated Cost Governance & Optimization
• Establish automated cost governance and optimization frameworks for data platforms.
• Improve cost visibility through telemetry, alerts, and policy-based controls.
• Drive proactive cost management rather than reactive spend reviews.

Required Skills & Experience

• Strong experience in Data Operations, Data Platform Operations, or SRE/DataOps roles.
• Proven experience reducing operational toil through automation-first approaches.
• Hands-on experience with incident management, observability, and operational tooling.
• Experience operating within or transforming managed services or vendor-led operating models.
• Strong understanding of cost governance and optimization in data platforms.
• Hands-on experience with Microsoft Azure data platform services such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Microsoft Fabric
• Experience implementing CI/CD and release automation using Azure DevOps, including Azure Pipelines, repository management, deployment workflows, and environment-based promotion controls.
• Experience with Microsoft Entra ID, Azure Key Vault, and Microsoft Purview for identity, secrets management, governance, data lineage, and compliance controls.
• Strong scripting and engineering skills in Python, SQL, and PowerShell to build automation, operational runbooks, telemetry pipelines, and remediation workflows.
• 8+ years of relevant experience.