Role: Network and Compute Engineer, AI Infrastructure
Location – San Jose, CA
Duration: Contract
Job Description: Network and Compute Engineer, AI Infrastructure
Position Summary
Network and Compute Engineer to design, deploy, validate, and operate high-performance infrastructure supporting enterprise artificial intelligence and accelerated-computing workloads.
The engineer will work across compute, networking, operating systems, container orchestration, cluster management, storage integration, security, and observability. The successful candidate will be comfortable working hands-on with complex infrastructure, diagnosing issues across multiple technical layers, and translating platform requirements into reliable and repeatable operational solutions.
The role is aligned to an NVIDIA AI Factory environment. The source material specifically covers NVIDIA Mission Control, Base Command Manager, Kubernetes installation, rack management, bare-metal management, observability, Run:ai, MGX architecture, Slurm, GB200 software and firmware, NVLink partitioning, XID troubleshooting, and telemetry.
Key Responsibilities
Compute and Cluster Engineering
· Deploy, configure, validate, and operate accelerated-compute clusters.
- Perform server and cluster bring-up, including operating-system imaging, boot validation, firmware alignment, hardware discovery, and post-deployment health checks.
- Support NVIDIA MGX-based infrastructure and understand the relationship between hardware architecture, firmware, operating systems, drivers, and the platform software stack.
- Install and administer Linux-based compute environments.
- Support Kubernetes installation, configuration, upgrades, and lifecycle operations.
- Deploy and administer Slurm-based workload environments.
- Assist with NVIDIA GB200 firmware updates, software-stack validation, sanity checks, and platform readiness testing.
- Support NVLink configuration and partitioning, where applicable.
- Troubleshoot GPU and system errors, including XID events, degraded nodes, boot failures, firmware inconsistencies, and workload-performance issues.
- Develop standardized, repeatable build and validation procedures for compute nodes and clusters.
The PDF explicitly identifies MGX architecture and software stack, MGX sanity checks, image creation, Slurm deployment, ARM images and booting, GB200 firmware and software-stack validation, NVLink partitioning, XID troubleshooting, and telemetry as relevant subject areas.
Network Engineering
· Design and support resilient network connectivity for accelerated-compute and AI cluster environments.
- Configure and troubleshoot high-speed Ethernet and, where applicable, InfiniBand fabrics.
- Support leaf-spine network architectures, network segmentation, routing, switching, addressing, name resolution, and time synchronization.
- Diagnose packet loss, latency, congestion, interface errors, cabling faults, routing issues, and workload connectivity failures.
- Validate host-to-host, node-to-storage, management-plane, and workload-plane connectivity.
- Support network configuration required for Kubernetes, cluster provisioning, telemetry, and distributed workloads.
- Work with security teams to implement network isolation, firewall rules, access controls, and secure administrative paths.
- Maintain accurate network diagrams, port maps, addressing plans, and configuration records.
Hiring note: High-speed Ethernet, InfiniBand, leaf-spine topology, firewall design, and related network technologies are recommended market requirements for this type of position. They are not individually specified in the uploaded PDF.
Platform Provisioning and Management
· Install, configure, and operate NVIDIA Base Command Manager.
- Support rack import, rack view, base view, node provisioning, and cluster lifecycle activities.
- Work with bare-metal management systems for server discovery, power operations, hardware health, firmware administration, and remote troubleshooting.
- Integrate cluster platforms with enterprise directory services.
- Support NVIDIA Mission Control processes and related AI Factory operational practices.
- Assist with the deployment and operation of Run:ai or equivalent workload-orchestration capabilities.
- Automate repetitive provisioning, validation, configuration, and reporting activities.
The source material explicitly includes Base Command Manager bring-up, directory services, rack import, rack view, base view, bare-metal management, observability, and Run:ai installation.
Monitoring, Reliability, and Operations
· Implement infrastructure monitoring for compute nodes, GPUs, network devices, interfaces, firmware, thermals, power, storage dependencies, and cluster services.
- Establish actionable dashboards, telemetry, alerts, and operational health checks.
- Lead technical diagnosis across hardware, firmware, Linux, Kubernetes, scheduler, network, and application layers.
- Perform root-cause analysis and document corrective and preventive actions.
- Develop operational runbooks for failure scenarios such as node loss, degraded network links, GPU errors, provisioning failures, and cluster-service disruption.
- Participate in incident response, problem management, capacity planning, lifecycle planning, and controlled change implementation.
- Identify automation and standardization opportunities that reduce manual effort, deployment defects, and recovery time.
- Work with infrastructure, cloud, security, storage, application, vendor, and service-management teams.
Security and Compliance Responsibilities
· Apply least-privilege access to management, compute, network, and orchestration platforms.
- Integrate administrative access with approved enterprise identity and directory services.
- Secure management interfaces, service accounts, automation credentials, application programming interfaces, and remote-access pathways.
- Maintain separation between management, workload, storage, and out-of-band management networks.
- Support firmware, operating-system, container, Kubernetes, and network vulnerability remediation.
- Ensure administrative actions and infrastructure changes are traceable through logging and change records.
- Protect sensitive cluster configuration, credentials, certificates, logs, and telemetry.
- Follow secure configuration standards, change controls, and applicable customer or regulatory requirements.
Hiring note: These security requirements are recommended additions for enterprise deployment. The PDF references directory services and hands-on operations, but it does not define a complete security or compliance control framework.
Required Technical Skills
· Strong administration and troubleshooting experience with Linux.
- Strong knowledge of data-center networking, including Ethernet, Transmission Control Protocol/Internet Protocol, routing, switching, virtual local area networks, link aggregation, Domain Name System, Dynamic Host Configuration Protocol, and Network Time Protocol.
- Hands-on experience installing or operating Kubernetes clusters.
- Experience deploying and supporting physical compute infrastructure.
- Understanding of server boot processes, operating-system imaging, firmware, device drivers, hardware management, and remote management controllers.
- Experience diagnosing issues across server, operating-system, network, and application layers.
- Working knowledge of Python, Bash, Ansible, or equivalent infrastructure-automation technologies.
- Experience with monitoring, logging, metrics, dashboards, alerting, and telemetry.
- Ability to create clear architecture diagrams, build documents, operating procedures, and troubleshooting runbooks.
- Strong analytical skills and the ability to work effectively during complex infrastructure incidents.
Preferred Technical Skills
· Experience with NVIDIA AI infrastructure, NVIDIA Mission Control, or NVIDIA Base Command Manager.
- Experience with NVIDIA MGX or GB200-based systems.
- Experience with Slurm and distributed-compute scheduling.
- Experience with Run:ai.
- Experience with InfiniBand, Remote Direct Memory Access, or Remote Direct Memory Access over Converged Ethernet.
- Knowledge of NVLink and multi-GPU communication.
- Experience interpreting NVIDIA XID errors and GPU telemetry.
- Experience with bare-metal provisioning and Redfish-compatible management interfaces.
- Experience with Prometheus, Grafana, OpenTelemetry, or equivalent observability platforms.
- Exposure to container runtimes, container registries, Kubernetes networking, and graphics-processing-unit operators.
- Experience supporting production AI, machine-learning, high-performance-computing, or large-scale data platforms.
Qualifications
· Bachelor’s degree in computer science, information technology, electronics, telecommunications, or a related field, or equivalent practical experience.
- Recommended 7 or more years of experience in infrastructure, compute, network, data-center, cloud-platform, or high-performance-computing engineering.
- Recommended 3 or more years of hands-on experience with Linux-based cluster platforms, Kubernetes, accelerated computing, or comparable large-scale infrastructure.
- Demonstrated experience owning complex technical problems from initial diagnosis through resolution and prevention.
