Overview
Skills
Job Details
Responsibilities
As a data engineer in the DE team, you will apply your strong technical experience building highly reliable services on managing and orchestrating multi-terabyte scale data lakes and implement a Data Mesh architecture, working closely with the Data Architecture/Modeling team.
Minimum Qualifications and Expectations:
At least 3 years (5 or 10 based on level) of professional experience as a software engineer or data engineer
A BS in Computer Science or equivalent experience
Strong programming skills (some combination of Python, Java, and Scala)
Experience writing SQL, structuring data, and data storage practices
Experience NoSQL databases like Mongodb and Cassandra
Experience with data modeling
Knowledge of data warehousing concepts
Experienced building data pipelines and micro services
Experience with Spark, Kafka, Flink, Hive, Airflow and other streaming and data pipeline technologies to process large volumes of streaming data
Experience working on Amazon Web Services (in particular using EMR, Kinesis, RedShift, S3, SQS and the like)
Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
An open mind to try solutions that may seem impossible at first
It's preferred, but not technically required, that you have:
Experience building self-service tooling and platforms
Built and designed Data Mesh architecture platforms
A passion for building and running continuous integration pipelines.
Built pipelines using Databricks and well versed with their API's
Contributed to open source projects (Ex: Operators in Airflow)
EDUCATION
Bachelor s Degree in Computer Science, Information Systems, Engineering or related field or equivalent work experience.