Overview
On Site
Full Time
Skills
Data-flow diagrams
Data structure
data lake
Data warehouse
Data loading
Data quality
Relational databases
PL/SQL
Cosmos-Db
Dimensional modeling
Star schema
Data
Microsoft Azure
Design
ELT
Extract
transform
load
Management
Database
Python
ADF
PostgreSQL
NoSQL
MongoDB
Writing
SQL
Snow flake schema
Job Details
Data Engineer
Responsibilities:
Build data pipelines / data orchestrator and data structures for existing operational data stores, Azure data lake & data warehouse.
Develop proof-of-concepts to evaluate new technologies and suggest improved and better ways to re-architect existing data loading processes and data structures.
Design and develop ELT / ETL to load data from internal sources into operational data stores, Azure data lake and data warehouse.
Create APIs, data structures and schemas to store and retrieve data.
Create and maintain data mappings, data definitions, architecture, and data flow diagrams.
Implement solutions to validate and test data quality.
Build proof-of-concepts to determine viability of new processes and technologies.
Deploy and manage code in non-prod and prod environments.
Troubleshoot data related issues and fix defects.
Optimize and performance tune databases and queries.
Participate in code reviews and train other team members.
Document code and create code templates based on industry best practices.
Requirements:
Bachelor's Degree and/or equivalent experience required.
Minimum 5 years of experience working as a Data Engineer, ETL Engineer, Data/ETL Architect or similar roles.
5+ years of solid continuous experience in Python.
3+ years of continuous experience with Airflow.
3+ years of experience with Azure Data Factory (ADF) or similar.
3+ years of experience working with relational databases: Oracle, SQL Server, PostgreSQL, or similar.
3+ years of experience working with NoSQL databases: MongoDB, Cosmos DB, DocumentDB or similar.
3+ years of experience writing SQL code.
3+ years of experience in Kimball Dimensional Modeling (Star-Schema).
2+ years of experience in columnar databases: Snowflake, Azure Synapse, or similar.
2+ years of Azure experience.
#RecruitPS
Responsibilities:
Build data pipelines / data orchestrator and data structures for existing operational data stores, Azure data lake & data warehouse.
Develop proof-of-concepts to evaluate new technologies and suggest improved and better ways to re-architect existing data loading processes and data structures.
Design and develop ELT / ETL to load data from internal sources into operational data stores, Azure data lake and data warehouse.
Create APIs, data structures and schemas to store and retrieve data.
Create and maintain data mappings, data definitions, architecture, and data flow diagrams.
Implement solutions to validate and test data quality.
Build proof-of-concepts to determine viability of new processes and technologies.
Deploy and manage code in non-prod and prod environments.
Troubleshoot data related issues and fix defects.
Optimize and performance tune databases and queries.
Participate in code reviews and train other team members.
Document code and create code templates based on industry best practices.
Requirements:
Bachelor's Degree and/or equivalent experience required.
Minimum 5 years of experience working as a Data Engineer, ETL Engineer, Data/ETL Architect or similar roles.
5+ years of solid continuous experience in Python.
3+ years of continuous experience with Airflow.
3+ years of experience with Azure Data Factory (ADF) or similar.
3+ years of experience working with relational databases: Oracle, SQL Server, PostgreSQL, or similar.
3+ years of experience working with NoSQL databases: MongoDB, Cosmos DB, DocumentDB or similar.
3+ years of experience writing SQL code.
3+ years of experience in Kimball Dimensional Modeling (Star-Schema).
2+ years of experience in columnar databases: Snowflake, Azure Synapse, or similar.
2+ years of Azure experience.
#RecruitPS