Java Spark Developer

Mandatory Skills:

• Strong experience in Java Spark, understanding of distributed computing, big data principles, and batch processing.
• Proficiency in understanding and working with AWS services, especially S3, Glue and Athena.

Detailed Description:

• Combine interface design concepts with digital design and establish milestones to encourage cooperation and teamwork.
• Develop overall concepts for improving the user experience within a business webpage or product, ensuring all interactions are intuitive and convenient for customers.
• Collaborate with back-end web developers and programmers to improve usability.
• Conduct thorough testing of user interfaces in multiple platforms to ensure all designs render correctly and systems function properly.
• Strong experience in Java Spark, understanding of distributed computing, big data principles, and batch processing. Proficiency in understanding and working with AWS services, especially S3, Glue and Athena.
• Understanding of Data Lake architectures and handling large volumes of structured and unstructured data. Good hands-on experience in Snowflake and SQL. Building scalable solutions to manage data ingestion, transformation, and storage in AWS-based Data Lake environments. Familiarity with various data formats.
• Candidate should be good in Python, who can create the python scripts to automate the data transformation processes.
• Required good experience in Unix Commands and Shell Scripting so he/she can create or enhance/enrich the existing shell scripts to add new functionality for the logging, error handling.
• Tools like IntelliJ (IDE), Confluence and Jira are must and Control-M (scheduling tool) is optional.

Good to have: