Job Summary
We are seeking an experienced Data Observability Architect to design and deliver scalable enterprise data & analytics platforms for utility clients.
The role will implement a Monte Carlo based Data Observability solution integrated with Snowflake, Exadata, SQL Server, dbt, Talend, Informatica, and Denodo to provide proactive monitoring, lineage, data quality, and SLA visibility. Enable automated anomaly detection, root cause analysis, impact assessment, ServiceNow-based alerting, operational dashboards, and knowledge transfer for sustainable operations.
Key Responsibilities
1. Architecture & Design
Design target state architecture for Observability solution that aligns with target state data pipeline framework
Integrate Monte Carlo observability framework with Snowflake, Exadata and SQL Server data platforms
Configure and enable Monte Carlo proactive monitoring components for Data freshness, Volume Anomalies, Schema drift, and Data quality
Configure and enable Monte Carlo predictive pipeline monitoring and issue detection for Snowflake and Dbt pipelines
Extend and implement Monte Carlo framework to support Talend ELT pipeline monitoring
Configure Monte Carlo SLA monitoring framework for Data timeliness, accuracy and availability
Configure operations lineage for traceability and analysis
Ensure Dashboards and reporting display data health KPIs and SLA performance metrics in Monte Carlo
Configure and test alerting and notification frameworks specific to data incidents, including severity classification, ownership, and routing leveraging ServiceNow
Integrate Monte Carlo observability framework with Informatica to analyze downstream blast radius
Pilot/ PoC - Extend and implement Monte Carlo framework to support Denodo virtualization layer monitoring, to be finalized during the project initiation phase
Enable root-cause-analysis through lineage identifying assets and pipelines contributing to failure incidents
Extend observability coverage to Talend ELT pipelines
Implement automated build, deployment, testing, rollback, and code-quality processes.
Support SonarQube integration and release governance
2. Data Engineering & Pipelines
Design and develop Talend-based ingestion pipelines supporting:
File-to-Database, Database-to-Database, API-to-Database, and Streaming-to-Database patterns
Build reusable and metadata-driven ingestion frameworks to enable rapid onboarding of new source systems and data formats
Integrate data from enterprise applications, databases, files, APIs, and event-streaming platforms
Developing Snowflake data pipelines, stored procedures, Snowpark applications, UDFs, automation frameworks, and performance-optimized ELT solutions
Strong expertise in Snowflake architecture and engineering, including Snowpipe, Snowpark, Streams & Tasks, Dynamic Tables, Cortex AI, performance optimization, data sharing, and ELT frameworks
3. Cloud & DevOps Integration
High Availability (HA), Disaster Recovery (DR), and Resiliency Design
Scalability Planning for Enterprise Data Platforms
Cloud Security Architecture and Compliance Controls
SSO Integration and Identity Management (Azure AD/Entra ID)
CI/CD Pipeline Design and Implementation
Git-Based Version Control and Branching Strategies
Automated Build, Testing, Deployment, and Release Management
4. Cost Optimized Design
Design and Implement consumption-aware observability framework along with dashboards for Snowflake usage and cost monitoring, workload monitoring, cost optimization
5. Security, Compliance and Governance
Role-based access control with Single Sign-On (SSO)
Explore and document potential Attribute-based access control capabilities
Audit logging and monitoring security events
Integration with Dominion Energy's existing privacy and security controls
6. Documentation and Knowledge Transfer
Architecture documentation (Reference architecture, Integration patterns, Data flow diagrams, Operational run books)
Prepare Technical design documents and Run books for observability monitoring
Knowledge transfer and enable internal teams to effectively monitor, manage, and maintain trusted data at scale
7. Stakeholder Engagement
Work with business teams across:
Grid & asset operations
Smart metering (AMI)
Billing & energy trading
Provide architectural leadership and mentoring
Required Skills & Experience
10-15 years of Data Engineering / Data Architecture experience
Hands-on experience with Monte Carlo Data Observability
Strong expertise in Snowflake (Snowpipe, Cortex and Cost Optimization & Consumption Observability), dbt, Talend, Informatica, SQL Server, and Oracle Exadata
Strong proficiency in Python scripting and automation for data engineering, observability, monitoring, API integration, metadata management, and operational support
Data Quality, Lineage, Data Governance, and SLA Monitoring experience
ServiceNow integration and enterprise monitoring framework
Strong stakeholder management and architecture leadership skills
Certifications (Preferred / Strongly Recommended)
Monte Carlo Customer Certification 101 (Preferred)
SnowPro Advanced Architect (ARA-C01)
SnowPro Core (COF-C02)
AWS Solutions Architect Professional (SAP-C02)
dbt Developer Certification
Nice to Have
Additional Certifications (Optional)
Fabric Data Engineer Associate (DP-700)
Cloud fundamentals (AWS / Google Cloud Platform basics)
ITIL v4 Foundation (ITIL4F)
CDMP, CISSP, ServiceNow CSA, Datadog, Splunk, Dynatrace, or Grafana Certifications
Python
Preferred Domain Experience
Utilities / Energy domain exposure:
Smart Metering (AMI)
Grid / SCADA systems
Energy trading / billing
Experience with:
Delta Lake & Unity Catalog
Event streaming (Kafka / Event Hub)
Domain Use Cases (Highly Relevant for Client)
Load forecasting
Demand response optimization
Asset performance monitoring
Outage prediction & grid resilience
Regulatory data reporting
Never miss an opportunity! Create an alert based on the job you applied for.
