AI Storage & HPC Infrastructure (100% Remote)
Dice Job Match Score™
🔢 Crunching numbers...
Job Details
Skills
- Storage
- Terraform
- HPC
- Lustre
Summary
About the Role
We are hiring 2-5 Production Engineers to join a high-impact Cloud Foundation AI Services team. You will build, scale, and operate the massive distributed storage infrastructure that powers AI researchers training cutting-edge models across cloud environments.
If you excel at solving complex I/O bottlenecks, optimizing throughput/latency, and scaling high-performance file systems for large-scale AI/HPC workloads, this role is for you.
Key Responsibilities
Storage Infrastructure: Build and operate distributed file and object storage systems specifically for AI training, shared researcher filesystems, massive datasets, and model checkpoints.
Performance Optimization: Diagnose and eliminate end-to-end storage bottlenecks across metadata, throughput, latency, caching, and network layers so storage I/O never stalls model training.
Automation & Orchestration: Automate the entire storage lifecycle (provisioning, scaling, upgrades, disaster recovery) using Kubernetes and Infrastructure-as-Code.
Data Migration: Execute seamless data movement and migrations across clusters, regions, and clouds without compromising data integrity or researcher velocity.
Reliability & SRE: Own production reliability through robust observability, alerting, automated testing, safe rollouts, and rapid incident response in collaboration with security and engineering teams.
Must-Have Qualifications
Production Storage Experience: Proven track record of building or operating production-grade distributed storage, high-performance file systems, or large-scale data infrastructure.
Linux & Debugging Expertise: Deep Linux engineering skills with the ability to debug complex filesystem, network, and concurrency issues in live production environments.
Coding / Automation: Strong software engineering or automation background writing maintainable code in languages such as Python, Go, C++, or Rust.
Ownership: Demonstrated history of owning reliability or performance improvement initiatives from initial detection through deployment and verification.
Nice-to-Have Skills (Bonus Points for Dice Search Filters)
Cloud & Orchestration: Experience with cloud infrastructure and Kubernetes or comparable large-scale orchestration engines.
Filesystem Internals: Depth in NFS/POSIX and distributed filesystem internals (metadata handling, caching layers, consistency models, and recovery).
Storage Systems: Hands-on experience with network/distributed storage vendors or technologies such as Hammerspace, Lustre, ZFS/OpenZFS, VAST, or Pure Storage.
AI/HPC Patterns: Familiarity with high-throughput data loading, checkpoint writes/restores, and staging data for high-performance accelerator clusters.
DevOps Tooling: Infrastructure-as-Code and observability stacks including Terraform, Helm, Kubernetes Operators, PrometheMetrics, and distributed tracing.
- Dice Id: 91173481
- Position Id: 9140794
- Posted 11 minutes ago
