Data Engineer

Data Engineer

Role Summary

The Senior Data Engineer is a hands-on expert and technical leader, actively engaged in designing, building, and optimizing scalable, reliable data pipelines at an enterprise level. This role not only guides architectural decisions but also directly implements advanced ELT solutions, troubleshoots complex data challenges, and ensures best practices through practical, high-impact contributions.

This role combines deep hands-on expertise with technical ownership, mentoring, and architectural alignment. The Senior Data Engineer drives and implements data engineering best practices, ensures high standards for quality and security, and partners with architecture and platform teams to improve the overall data ecosystem.

Required Skills & Experience

• Advanced expertise in SQL, ELT patterns, and performance tuning.
• Strong experience with Oracle Exadata, Snowflake or similar cloud/on-prem data warehouses.
• Hands-on experience with enterprise ETL/ELT platforms (e.g., Talend, dbt, Informatica).
• Deep understanding of data warehousing architecture and dimensional modeling.
• Experience designing and supporting large-scale, production data pipelines.
• Strong scripting experience (Python, shell).
• Experience with data virtualization tools (e.g., Denodo, Composite, Dremio, Starburst).
• Experience with DataOps practices, CI/CD, and observability.
• Required 5 to 7+ years of Data Engineering experience.
• ETL development and process support; may require weekend/off-business-hours work.

Key Responsibilities

• Build end-to-end data pipelines and ETL/ELT solutions to support analytics, reporting, and AI/ML use cases.
• Apply scalable patterns for batch and incremental processing by developing, testing, and deploying data workflows.
• Review and implement data modeling, transformation logic, and performance strategies.
• Evaluate, select, and integrate tooling, frameworks, and platform capabilities.
• Build complex, high-volume data pipelines using SQL-centric ETL/ELT patterns.
• Design and implement scalable streaming pipelines to process real-time data.
• Lead performance tuning efforts across pipelines, warehouses, and workloads.
• Ensure data pipelines are resilient, observable, and production ready.
• Implement enterprise-grade error handling, restartability, and monitoring.
• Build and maintain scalable, low-latency streaming data pipelines using Kafka, Kinesis, or Spark Streaming.
• Perform on-the-fly data cleaning, validation, and enrichment.
• Utilize indexing and partitioning strategies to optimize warehouse and big data environments.
• Implement standards for data quality checks, validation, and reconciliation.
• Ensure pipelines meet security, access control, and governance requirements.
• Partner with governance and DataOps teams on metadata, lineage, and auditability.
• Improve operational monitoring, alerting, and incident response processes.
• Identify reliability, performance, and cost optimization opportunities.
• Support production troubleshooting and root cause analysis.
• Investigate data quality incidents and identify design/coding gaps.
• Participate in design and code reviews.
• Partner with infrastructure teams, application teams, and architects on complex transformations.
• Translate ambiguous requirements into technical solutions.
• Work across complex multi-platform environments.

Behavioral Expectations

• Delivering at Pace
• Collaborative Teamwork
• Effective Communication
• Ownership and Adaptability
• Ability to Work Independently
• Achievement Orientation
• Self-Starter
• Concern for Quality
• Flexibility

Preferred Qualifications

• Experience supporting AI/ML or advanced analytics pipelines.
• Cloud platform experience (AWS, Azure, or GCP).
• Prior experience influencing enterprise data standards or reference architecture.
• Experience optimizing cost and performance in cloud data warehouses.
• Hands-on experience with Cribl, Apache Kafka, Kafka Connect, Spark Streaming, or Apache Flink.

Education

• Bachelor''s Degree or higher required
• Computer Science, Information Systems, Mathematics, or related discipline

Preferred Industry Background

• High preference for candidates with large-scale utility industry experience.
• Will also consider candidates supporting large-scale capital projects.