Data Engineer

Job Summary

Location: United States

Experience: 5+ years in Data Engineering

Job Type: Permanent

Thinking Minds' client is looking for an experienced Data Engineer to expand and optimize their data and data pipeline architecture. This role is crucial for optimizing data flow and collection across cross-functional teams. The successful candidate will support software developers, data architects, and data analysts in data initiatives, enhancing the company's data warehouse to support next-generation products.

Ideal Candidate Should Have

Experience with ETL processes

Proficiency in SQL and complex query writing

Strong understanding of data warehousing

Ability to work in mixed cloud and on-prem environments

Skills in data integration and transformation

Role Overview

The Data Engineer will be responsible for building both real-time and batch data integrations from various source systems into the enterprise data warehouse. They will work closely with data and analytics experts to enhance the functionality of data systems. The role involves assembling large, complex data sets to meet business requirements, designing and implementing real-time ETL and batch processing, and providing production support for data pipelines.

Key Responsibilities

Build and optimize data pipelines and databases

Write complex SQL queries for reporting

Assemble large, complex data sets

Design and implement real-time ETL processes

Provide production support for data integration

Scale and tune data pipelines

Develop algorithms to improve data utility

Collaborate with data architects and analysts

Top Skills and Modules

ETL processes

SQL

Data warehousing

Data integration

Real-time and batch processing

Data pipeline optimization

Cloud and on-prem environments