Our client is currently seeking a Data Lake Software Engineer
This job will have the following responsibilities:
The Data Lake will be built on the AWS Cloud, with pipelines implemented in a mixture of PySpark and SparkSQL triggered from a REST API.
* This will require someone who has hands-on experience developing systems that leverage multiple AWS services including CloudFormation, Lambda, Glue and API Gateway for data management and analysis.
* Language requirements: Python, PySpark, SQL.
* Experience migrating legacy systems into an Enterprise Data Lake is a plus.
* All infrastructure will deployed using scripted processes developed with Agile methodologies.
* Software Engineers will be responsible for building and maintaining both the Data Lake infrastructure and multiple ETL pipelines that run on it.