Role : Network and Compute Engineer, AI Infrastructure
Location : San Jose, CA or Anywhere in US and ready to travel
Note: Client is not willing to sponsor any visa.
Job Description: Network and Compute Engineer, AI Infrastructure
Position Summary
Network and Compute Engineer to design, deploy, validate, and operate high-performance infrastructure supporting enterprise artificial intelligence and accelerated-computing workloads.
The engineer will work across compute, networking, operating systems, container orchestration, cluster management, storage integration, security, and observability. The successful candidate will be comfortable working hands-on with complex infrastructure, diagnosing issues across multiple technical layers, and translating platform requirements into reliable and repeatable operational solutions.
The role is aligned to an NVIDIA AI Factory environment. The source material specifically covers NVIDIA Mission Control, Base Command Manager, Kubernetes installation, rack management, bare-metal management, observability, Run:ai, MGX architecture, Slurm, GB200 software and firmware, NVLink partitioning, XID troubleshooting, and telemetry.
Key Responsibilities
Compute and Cluster Engineering
· Deploy, configure, validate, and operate accelerated-compute clusters.
Perform server and cluster bring-up, including operating-system imaging, boot validation, firmware alignment, hardware discovery, and post-deployment health checks.
Support NVIDIA MGX-based infrastructure and understand the relationship between hardware architecture, firmware, operating systems, drivers, and the platform software stack.
Install and administer Linux-based compute environments.
Support Kubernetes installation, configuration, upgrades, and lifecycle operations.
Deploy and administer Slurm-based workload environments.
Assist with NVIDIA GB200 firmware updates, software-stack validation, sanity checks, and platform readiness testing.
Support NVLink configuration and partitioning, where applicable.
Troubleshoot GPU and system errors, including XID events, degraded nodes, boot failures, firmware inconsistencies, and workload-performance issues.
Develop standardized, repeatable build and validation procedures for compute nodes and clusters.
The PDF explicitly identifies MGX architecture and software stack, MGX sanity checks, image creation, Slurm deployment, ARM images and booting, GB200 firmware and software-stack validation, NVLink partitioning, XID troubleshooting, and telemetry as relevant subject areas.
Network Engineering
· Design and support resilient network connectivity for accelerated-compute and AI cluster environments.
Configure and troubleshoot high-speed Ethernet and, where applicable, InfiniBand fabrics.
Support leaf-spine network architectures, network segmentation, routing, switching, addressing, name resolution, and time synchronization.
Diagnose packet loss, latency, congestion, interface errors, cabling faults, routing issues, and workload connectivity failures.
Validate host-to-host, node-to-storage, management-plane, and workload-plane connectivity.
Support network configuration required for Kubernetes, cluster provisioning, telemetry, and distributed workloads.
Work with security teams to implement network isolation, firewall rules, access controls, and secure administrative paths.
Maintain accurate network diagrams, port maps, addressing plans, and configuration records.
Hiring note: High-speed Ethernet, InfiniBand, leaf-spine topology, firewall design, and related network technologies are recommended market requirements for this type of position. They are not individually specified in the uploaded PDF.
Platform Provisioning and Management
· Install, configure, and operate NVIDIA Base Command Manager.
Support rack import, rack view, base view, node provisioning, and cluster lifecycle activities.
Work with bare-metal management systems for server discovery, power operations, hardware health, firmware administration, and remote troubleshooting.
Integrate cluster platforms with enterprise directory services.
Support NVIDIA Mission Control processes and related AI Factory operational practices.
Assist with the deployment and operation of Run:ai or equivalent workload-orchestration capabilities.
Automate repetitive provisioning, validation, configuration, and reporting activities.
The source material explicitly includes Base Command Manager bring-up, directory services, rack import, rack view, base view, bare-metal management, observability, and Run:ai installation.
Monitoring, Reliability, and Operations
· Implement infrastructure monitoring for compute nodes, GPUs, network devices, interfaces, firmware, thermals, power, storage dependencies, and cluster services.
Establish actionable dashboards, telemetry, alerts, and operational health checks.
Lead technical diagnosis across hardware, firmware, Linux, Kubernetes, scheduler, network, and application layers.
Perform root-cause analysis and document corrective and preventive actions.
Develop operational runbooks for failure scenarios such as node loss, degraded network links, GPU errors, provisioning failures, and cluster-service disruption.
Participate in incident response, problem management, capacity planning, lifecycle planning, and controlled change implementation.
Identify automation and standardization opportunities that reduce manual effort, deployment defects, and recovery time.
Work with infrastructure, cloud, security, storage, application, vendor, and service-management teams.
Security and Compliance Responsibilities
· Apply least-privilege access to management, compute, network, and orchestration platforms.
Integrate administrative access with approved enterprise identity and directory services.
Secure management interfaces, service accounts, automation credentials, application programming interfaces, and remote-access pathways.
Maintain separation between management, workload, storage, and out-of-band management networks.
Support firmware, operating-system, container, Kubernetes, and network vulnerability remediation.
Ensure administrative actions and infrastructure changes are traceable through logging and change records.
Protect sensitive cluster configuration, credentials, certificates, logs, and telemetry.
Follow secure configuration standards, change controls, and applicable customer or regulatory requirements.
Hiring note: These security requirements are recommended additions for enterprise deployment. The PDF references directory services and hands-on operations, but it does not define a complete security or compliance control framework.
Required Technical Skills
· Strong administration and troubleshooting experience with Linux.
Strong knowledge of data-center networking, including Ethernet, Transmission Control Protocol/Internet Protocol, routing, switching, virtual local area networks, link aggregation, Domain Name System, Dynamic Host Configuration Protocol, and Network Time Protocol.
Hands-on experience installing or operating Kubernetes clusters.
Experience deploying and supporting physical compute infrastructure.
Understanding of server boot processes, operating-system imaging, firmware, device drivers, hardware management, and remote management controllers.
Experience diagnosing issues across server, operating-system, network, and application layers.
Working knowledge of Python, Bash, Ansible, or equivalent infrastructure-automation technologies.
Experience with monitoring, logging, metrics, dashboards, alerting, and telemetry.
Ability to create clear architecture diagrams, build documents, operating procedures, and troubleshooting runbooks.
Strong analytical skills and the ability to work effectively during complex infrastructure incidents.
Preferred Technical Skills
· Experience with NVIDIA AI infrastructure, NVIDIA Mission Control, or NVIDIA Base Command Manager.
Experience with NVIDIA MGX or GB200-based systems.
Experience with Slurm and distributed-compute scheduling.
Experience with Run:ai.
Experience with InfiniBand, Remote Direct Memory Access, or Remote Direct Memory Access over Converged Ethernet.
Knowledge of NVLink and multi-GPU communication.
Experience interpreting NVIDIA XID errors and GPU telemetry.
Experience with bare-metal provisioning and Redfish-compatible management interfaces.
Experience with Prometheus, Grafana, OpenTelemetry, or equivalent observability platforms.
Exposure to container runtimes, container registries, Kubernetes networking, and graphics-processing-unit operators.
Experience supporting production AI, machine-learning, high-performance-computing, or large-scale data platforms.
Qualifications
· Bachelor’s degree in computer science, information technology, electronics, telecommunications, or a related field, or equivalent practical experience.
Recommended 7 or more years of experience in infrastructure, compute, network, data-center, cloud-platform, or high-performance-computing engineering.
Recommended 3 or more years of hands-on experience with Linux-based cluster platforms, Kubernetes, accelerated computing, or comparable large-scale infrastructure.
Demonstrated experience owning complex technical problems from initial diagnosis through resolution and prevention.
Hiring note: The experience thresholds and educational requirements above are suggested market-screening criteria. They are not specified in the source PDF and can be adjusted for Engineer, Senior Engineer, Lead, or Architect levels.
Relevant Certifications
Certifications are desirable but should not replace proven hands-on capability:
· NVIDIA Certified Associate or NVIDIA Certified Professional credentials relevant to AI infrastructure
Red Hat Certified System Administrator or Red Hat Certified Engineer
Certified Kubernetes Administrator
Cisco Certified Network Professional, Data Center or Enterprise
NVIDIA networking or InfiniBand training
Linux Foundation certifications
Security-focused credentials appropriate to the operating environment
Behavioral Competencies
· Strong ownership and structured problem-solving.
Willingness to diagnose problems independently and validate solutions through hands-on testing.
Ability to work across organizational and technology boundaries.
Clear communication with engineering teams, leadership, customers, partners, and vendors.
Ability to explain complex infrastructure issues in plain business language.
Disciplined approach to documentation, change control, risk management, and operational handover.
Continuous-learning mindset in a rapidly evolving technology environment.
The PDF emphasizes hands-on learning, foundational Linux, Kubernetes and networking knowledge, independent problem-solving, collaboration, and the ability to represent the platform effectively to customers and partner organizations.
Measures of Success
· Successful, repeatable cluster and node deployments.
High availability and operational stability of network and compute platforms.
Reduced build time through automation and reusable configurations.
Faster detection, diagnosis, and recovery from infrastructure failures.
Lower configuration drift and fewer manual deployment errors.
Complete operational documentation and effective knowledge transfer.
Timely remediation of security vulnerabilities and unsupported firmware or software.
Efficient use of expensive accelerated-compute capacity.
Positive customer and stakeholder outcomes during deployments and production operations.
Recommended Interview Focus
Candidates should be assessed through scenario-based and hands-on questions rather than only theoretical discussion:
1. Cluster bring-up: Explain how you would isolate a compute node that boots but does not join the cluster.
2. Network diagnosis: Diagnose intermittent performance degradation across a group of distributed GPU nodes.
3. Kubernetes: Explain how you would investigate a workload that requests GPUs but remains unscheduled.
4. Firmware and drivers: Describe how you would manage a firmware or driver mismatch across a production cluster.
5. GPU troubleshooting: Walk through the investigation of a recurring XID event.
6. Automation: Provide an example of automating provisioning, validation, or configuration compliance.
7. Security: Design secure access across the management, workload, storage, and out-of-band networks.
8. Operations: Define what should be captured in telemetry and alerting for an accelerated-compute cluster.
9. Customer communication: Explain a major infrastructure issue to a business stakeholder without relying on specialist terminology.
Suggested Job-Posting Title Options
· Senior Network and Compute Engineer, AI Infrastructure
· AI Factory Network and Compute Engineer
· GPU Cluster Infrastructure Engineer
· Network and Accelerated Compute Platform Engineer
· NVIDIA AI Infrastructure Engineer
