Overview
On Site
Depends on Experience
Contract - W2
Contract - Independent
Contract - 12 Month(s)
No Travel Required
Skills
Apache Airflow
Big Data
Data Engineering
Data Lakehouse
Data LakehouseDesign
Engineer designing
ETL
PLSQL
Pytest
Python
Scikit - Learn
data integration
data pipeline
data pipelines
midstream
pandas
performance tuning
Numpy
Kubernetes
Amazon S3
Physical Data Model
SQL
Software Development
Data Quality
Extract
Transform
Load
ELT
Code Refactoring
Continuous Delivery
Continuous Integration
Testing
Use Cases
scikit-learn
Advanced Analytics
Job Details
Client is currently seeking an experienced Data Engineer to join the Big Data and Advanced Analytics department. The Data Engineer will work closely with business domain experts to support data analytic use cases for the midstream oil and gas operations, engineering, and measurements business units.
Responsibilities include:
- Design and implement reliable data pipelines to integrate disparate data sources into a single Data Lakehouse
- Design and implement data quality pipelines to ensure data correctness and building trusted datasets
- Assist with data platform performance tuning and physical data model support including partitioning and compaction
- Provide guidance in data visualizations and reporting efforts to ensure solutions are aligned to business objectives
The successful candidate will meet the following qualifications:
- 5+ years of experience as a Data Engineer designing and maintaining data pipeline architectures
- 5+ years of programming experience in Python, ANSI SQL, PLSQL, and TSQL
- Experience in various data integration patterns including ETL, ELT, Pub/Sub, and Change Data Capture
- Experience with common Python Data Engineering packages including pandas, Numpy, Pyarrow Pytest, Scikit-Learn, and Boto3
- Experience in software development practices such as Design Principles and Patterns, Testing, Refactoring, CI/CD, and version control
- Knowledgeable of modern data platform technologies including Apache Airflow, Kubernetes, and S3 Object Storage
- Experience with Dremio and Airbyte is preferred
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.