We are seeking a Senior Data Software Engineer to implement monitoring, incident management, and operational capabilities for an automated Data Platform Monitoring solution. In this role, you will design and maintain robust monitoring rules, incident automation, and operator health views to ensure the reliability and observability of data products and pipelines. Responsibilities Implement monitoring rules for data pipeline health, freshness, volume, run outcomes, and other agreed monitoring objectives Develop data processing and transformation logic for monitoring events and operational data Build automated Jira incident creation based on detected monitoring events Ensure incidents contain the required information, including component, region, DAG/job, run, timestamp, detecting check, error reference, and severity Support incident recurrence handling, stakeholder notification, and monitoring of incident status Develop and maintain the operator workspace/health view for monitoring data products and pipeline status Enable integration of monitoring data with Snowflake and other components of the solution Participate in testing monitoring rules and incident automation against real and historical failures Troubleshoot issues, improve monitoring logic, and support solution handover Requirements 3+ years of experience in data engineering roles Expertise in Apache Airflow, Snowflake, and dbt Proficiency in building and maintaining data pipelines, transformation logic, and operational data flows Background in implementing monitoring, observability, and alerting for data platforms Skills in incident automation and integration with tools such as Jira Familiarity with applying AI capabilities within data platform or monitoring solutions Understanding of pipeline health metrics, including freshness, volume, and run outcomes Capability to troubleshoot complex data platform issues and drive continuous improvement English proficiency at an Upper-Intermediate level (B2) or higher
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: 10330481
- Position Id: 293f5e942e93cd9914348582828697a
- Posted 4 hours ago
Never miss an opportunity! Create an alert based on the job you applied for.
