Linux HPC Systems Engineer

Requisition Id 14831

Overview:

This position requires regular on-site presence.

We are hiring a Linux HPC Engineer who will focus on supporting researcher's high-performance computing (HPC), data engineering and management, and new technology needs! We enable research while protecting ORNL assets, especially from cyber incidents. Our team works with other clustered computing and HPC groups to help research programs identify the best solutions for their needs. When we build environments for our customers, to the Linux HPC Systems Engineer designs, implements and maintains the systems from inception to retirement. This position resides in the Emerging Technologies, AI & Computing group in the Research Computing Support division in the Information Technology Services Directorate at Oak Ridge National Laboratory (ORNL).

This role advocates and promotes HPC and clustered computing services to researchers who process large data sets and/or develop code as a part of their project. This includes ensuring the availability, performance, scalability, and security of production systems. The ETAC Group frequently uses automation and monitoring solutions to minimize our day-to-day maintenance and are always looking for opportunities to optimize system management practices or system performance. As the primary domain experts for these systems, you will work with technical staff to install and help tune the performance of various scientific toolsets.

Major Duties/Responsibilities:

• Develop, architect, and engineer HPC systems and related application software solutions, including HPC research projects. This includes:
  • Monitoring for system issues.
  • Guiding project tasks through the HPC engineering processes and ensuring that standard methodologies are implemented continuously and consistently.
  • Managing backup services.
  • Troubleshooting and resolving HPC system problems quickly and effectively.
  • Working with other HPC systems engineers and vendors to resolve hardware and software issues.
• Create and foster partnerships with HPC research at ORNL to encourage outstanding delivery of services.
• Translate project deliverables, milestones, and timelines into predictable team output and tasks.
• Provide consulting in the selection and purchase of hardware and software systems.
• Ensure HPC configuration management using tools such as Git, Jenkins, Ansible, Puppet, etc.
• Ensure the secure and effective operation of HPC systems through compliance with ORNL procedures and IT Internal Operating Procedures.
• Develop and document system and service diagrams, procedures, and software build/install notes.
• Deliver ORNL's mission by aligning behaviors, priorities, and interactions with our core values of Impact, Integrity, Teamwork, Safety, and Service. Promote equal opportunity by fostering a respectful workplace - in how we treat one another, work together, and measure success.

Basic Qualifications:

• A BS degree in computer science, computer engineering, information technology, information systems, science, engineering, business, or a related discipline and a minimum of five (5) to seven (7) years of aligned professional experience is required for consideration. An overall combination of equivalent education and experience may be considered.
• Masters and PhD degree holders in the same fields of study are also encouraged to apply:
  • Masters' holders should have a minimum of four (4) to six (6) years of relevant and aligned experience.
  • PhD holders should have up to three (3) years of relevant and aligned experinece.
• Experience monitoring and maintaining HPC hardware and software including, but not limited to, InfiniBand, Slurm, Lustre, RDMA, Weka, and related technologies central to this team's work.
• Experience with configuration management and automation tools such as Git, Jenkins, Ansible, Puppet.
• Experience creating scripts using Bash, Python, etc.

Preferred Qualifications:

• Experience supporting AI-enabled HPC systems and software for model training is heavily preferred.
• Understanding of HPC platforms to support users with job submissions and troubleshooting.
• Excellent interpersonal skills suitable for communication with customers and management.
• Effective written, presentation, and verbal communication skills.
• Experience with Centos/RHEL, Ubuntu, VMware.
• Experience building and running containerized applications in an HPC environment. Knowledge of Apptainer, Warewulf, Fuzzball.
• Experience managing systems using GPU/CUDA clusters for AI/ML and/or image processing.
• Proven ability to work in a dynamic environment and supporting large data systems.
• Effective documentation skills, including ability to prepare simple documentation web pages.

Special Requirements:

Visa sponsorship is not available for this position.

This position requires the ability to obtain and maintain a clearance from the Department of Energy. As such, this position is a Workplace Substance Abuse (WSAP) testing designated position. WSAP positions require passing a pre-placement drug test and participation in an ongoing random drug testing program.

About ORNL:

As a U.S. Department of Energy (DOE) Office of Science national laboratory, ORNL has an impressive 80-year legacy of addressing the nation's most pressing challenges. Our team is made up of over 7,000 dedicated and innovative individuals! Our goal is to create an environment where a variety of perspectives and backgrounds are valued, ensuring ORNL is known as a top choice for employment. These principles are essential for supporting our broader mission to drive scientific breakthroughs and translate them into solutions for energy, environmental, and security challenges facing the nation.

ORNL offers competitive pay and benefits programs to attract and retain individuals who demonstrate exceptional work behaviors. The laboratory provides a range of employee benefits, including medical and retirement plans and flexible work hours, to support the well-being of you and your family. Employee amenities such as on-site fitness, banking, and cafeteria facilities are also available for added convenience.

Other benefits include the following: Prescription Drug Plan, Dental Plan, Vision Plan, 401(k) Retirement Plan, Contributory Pension Plan, Life Insurance, Disability Benefits, Generous Vacation and Holidays, Parental Leave, Legal Insurance with Identity Theft Protection, Employee Assistance Plan, Flexible Spending Accounts, Health Savings Accounts, Wellness Programs, Educational Assistance, Relocation Assistance, and Employee Discounts.

If you have difficulty using the online application system or need an accommodation to apply due to a disability, please email:

#LI-DC1

This position will remain open for a minimum of 5 days after which it will close when a qualified candidate is identified and/or hired.

We accept Word (.doc, .docx), Adobe (unsecured .pdf), Rich Text Format (.rtf), and HTML (.htm, .html) up to 5MB in size. Resumes from third party vendors will not be accepted; these resumes will be deleted and the candidates submitted will not be considered for employment.

If you have trouble applying for a position, please email

ORNL is an equal opportunity employer. All qualified applicants, including individuals with disabilities and protected veterans, are encouraged to apply. UT-Battelle is an E-Verify employer.