- Strong real-life experience in python development especially in PySpark in AWS Cloud environment.
- Design and develop data ingest and transform processes
- Minimum 3 years of relevant experience, in PySpark in AWS Cloud environment
- Proficient at Python Spark and the AWS ecosystem
- Experience in Python and common python libraries.
- Experience with AWS Cloud on data integration with Apache Spark, EMR, Glue, Kafka, Kinesis, and Lambda in S3, Redshift, RDS
- Knowledge of Big Data, data engineering, databases, data wrangling, relational databases
- Experience in Design, develop test, deploy, maintain and improving data integration pipeline.