Python HPC Pipeline Engineer

Role: Python HPC Pipeline Engineer

Location: Remote

Description:

We are looking for an expert developer who is highly skilled with Python. Primary focus will be developing and maintaining the HPC pipelines, Write unit test cases, Interact with teams in different timezone, migration of Python pipeline from Google Cloud Platform to HPC clusters, following generally accepted practices and workflows. Ensuring that you produce robust, secure, modular, and maintainable code. You will coordinate with other team members, including backend, Devops engineers. Your commitment to team collaboration, perfect communication, and a quality product is crucial.

Qualifications and Skills:

• B Tech, MS or PhD degree in Computer Science or similar.
• 5-8 years of strong hands-on experience in Python.
• Strong hands-on experience in Bash scripting.
• Strong knowledge of Docker, Google Cloud Clusters (Compute engine, GCS)
• Should have working experience of on-prem cluster.
• Good to have knowledge about HPC Operations work.
• Good communication skills and ability to work independently
• Expertise in understanding and analyzing requirements
• Participate in pair programming and peer reviews. You'll collaborate with other engineers on problems of all shapes and sizes, from code to new ideas and methodologies.
• Incorporate automated testing into development and maintenance procedures.
• Strong Knowledge of data structure, functional programming and object-oriented programming paradigms
• Ability to write efficient, secure, well-documented, and clean Python code
• Proficiency with modern development tools, like Git
• Experience with both consuming and designing Pipelines setups.
• Suggest any enhancements or changes that are required to stay up with modern security and development best practices.
• Good to have Google Cloud Platform.

Responsibilities:

• Designs, analyzes, and documents cluster pipelines
• Investigates and analyzes verbal and written requests for pipelines.
• Excellent teamwork and communication abilities
• Write backend code/scripts in programming languages like Python / BASH
• Maintains high standards of quality for code, functional specification documentation, and deliverables
• Self-motivated and self-managing, with strong organizational skills
• Ability to work with tight deadlines and multiple competing priorities
• Write efficient, secure, clean, scalable and robust Python code that is effective
• Test and troubleshoot the pipeline to ensure its performance.
• Ability to optimize the pipeline for performance
• Interact with development teams to develop a strong understanding of the project and testing objectives.
• Participate in troubleshooting of issues with different teams to drive towards root cause identification and resolution
• Documentation skills to track the development and implementations
• Effective communication skills: Regularly achieve consensus with peers, and clear status updates.

Desired Skills:

• Python [Must]
• Bash scripting [Must]
• Docker [Must]
• HPC Operations [Must]
• LSF jobs, DDN s understanding [Must]
• Google Cloud Platform / HPC clusters knowledge [Must]
• Data Structures / Object oriented programming concepts [Must]
• Experience in building Google cloud solutions and/or microservices. [optional]
• Knowledge of Git [Must]