Apply Now

Member of Technical Staff (Software Engineer, Inference & Training Platform)

Remote in New York, NY, US • Posted 19 hours ago • Updated 6 hours ago

Full Time

On-site

USD $250,000.00 - 485,000.00 per year

Fitment

Dice Job Match Score™

📋 Comparing job requirements...

Job Details

Skills

Artificial Intelligence
Real-time
Cloud Computing
Provisioning
Lifecycle Management
Orchestration
Management
Technical Direction
IaaS
Roadmaps
Expect
Kubernetes
Computer Hardware
Computer Networking
Amazon Web Services
Google Cloud
Google Cloud Platform
Scheduling
Resource Allocation
Rust
C++
Value Engineering
Training
High Availability
Stacks Blockchain
HPC
GPU
CUDA
InfiniBand
Remote Direct Memory Access
Machine Learning (ML)
Grafana

Artificial Intelligence
Real-time
Cloud Computing
Provisioning
Lifecycle Management
Orchestration
Management
Technical Direction
IaaS
Roadmaps
Expect
Kubernetes
Computer Hardware
Computer Networking
Amazon Web Services
Google Cloud
Google Cloud Platform
Scheduling
Resource Allocation
Rust
C++
Value Engineering
Training
High Availability
Stacks Blockchain
HPC
GPU
CUDA
InfiniBand
Remote Direct Memory Access
Machine Learning (ML)
Grafana

Summary

Perplexity serves hundreds of millions of queries a month, and every one of them fans out into multiple AI inference requests running in real time. Behind that sits a large GPU fleet spread across several cloud providers. Today, our inference engineers and researchers build models while also managing networking, securing capacity, and operating the underlying GPU clusters, responsibilities we want a dedicated platform team to own. Your job is to take ownership of that infrastructure and hide its complexity behind a unified, self-serve platform for running training and inference workloads.

Responsibilities

Build a self-serve compute platform. Design and own the systems that let inference engineers and researchers launch training jobs and operate inference services without managing GPU provisioning, cluster configuration, or provider-specific infrastructure.
Operate the GPU fleet. Own provisioning, lifecycle management, reliability, and capacity integration across providers, giving teams a consistent way to use compute regardless of where it runs.
Solve for GPU scarcity. Build the scheduling and placement logic that finds available capacity across providers, packs it efficiently, and gets the right workload onto the right hardware under real constraints.
Support two very different workloads. Keep long-running distributed training jobs healthy while simultaneously guaranteeing the availability and latency of production inference services on the same fleet.
Own the Kubernetes for GPU orchestration. Write the operators and CRDs, and manage many clusters across providers so the platform behaves the same everywhere we run.
Make failure boring. Build the fault tolerance, autoscaling, and observability that keep the fleet utilized and let workloads survive node loss, provider hiccups, and capacity shifts without human intervention.
Set technical direction across teams. Partner with inference and cloud infrastructure engineers to turn operational constraints into a coherent platform architecture and roadmap.

Qualifications

We expect you to have real depth in most of these:

Deep Kubernetes experience - custom operators, CRDs, and multi-cluster federation, not just running kubectl apply.
You've managed GPU clusters at scale: NVIDIA hardware, CUDA, and the networking that makes them fast (InfiniBand or RoCE).
You've orchestrated compute across multiple clouds (CoreWeave, AWS, Google Cloud Platform, or similar) and understand how different each one really is.
Strong distributed systems fundamentals: scheduling, resource allocation, and fault tolerance under load.
You write infrastructure and systems-level code in Go, Rust or C++.
You've supported both long-running training jobs and high-availability inference services, and you know why they pull infrastructure in opposite directions.
You own problems end-to-end and do well when the path forward isn't laid out for you.

Additional experience we value

Inference serving stacks: vLLM, SGLang, or TensorRT-LLM.
Slurm or other HPC schedulers.
GPU kernel work in CUDA or Triton - not required, but notable.
High-speed interconnects: InfiniBand, RoCE, or RDMA in production.
Observability for ML workloads: Prometheus, Grafana, or Weights & Biases.

If you're excited about this role, we encourage you to apply even if your experience doesn't match every qualification listed above.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91140125
Position Id: 83daab61098bdabe0ec1080f33783f94
Posted 19 hours ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Senior Staff Solutions Engineer (NYC)

New York, New York

•

Today

Crusoe is on a mission to accelerate the abundance of energy and intelligence. As the only vertically integrated AI infrastructure company built from the ground up, we own and operate each layer of the stack - from electrons to tokens - to power the world's most ambitious AI workloads. When you join Crusoe, you join a team that is building the future, faster. We're in the midst of the greatest industrial revolution of our time. The demand for AI compute is boundless, and power is a bottleneck.

Full-time

USD 175,000.00 - 250,000.00 per year

Staff Platform Engineer, Core Cloud Platform

New York, New York

•

Today

About AlphaSense: The world's most sophisticated companies rely on AlphaSense to remove uncertainty from decision-making. With market intelligence and search built on proven AI, AlphaSense delivers insights that matter from content you can trust. Our universe of public and private content includes equity research, company filings, event transcripts, expert calls, news, trade journals, and clients' own research content. The acquisition of Tegus by AlphaSense in 2024 advances our shared mission to

Full-time

USD 223,100.00 - 305,000.00 per year

Staff Infrastructure Engineer

New York, New York

•

Today

THE POSITIONOur roster has an opening with your name on it As a Staff Infrastructure Engineer, you'll own the design, operation, and maturity of our Kubernetes platforms and service mesh ecosystem. We run 100+ Kubernetes clusters across AWS Cloud and Outposts. Kong Mesh sits at the heart of how our services talk to each other - securing traffic, managing policies, handling retries and timeouts. You'll bring expertise on Kubernetes and mesh and help us operate these platforms with confidence and

Full-time

USD 159,000.00 - 208,950.00 per year

Staff Infrastructure Engineer

New York, New York

•

8d ago

THE POSITION Our roster has an opening with your name on it As a Staff Infrastructure Engineer, you'll own the design, operation, and maturity of our Kubernetes platforms and service mesh ecosystem. We run 100+ Kubernetes clusters across AWS Cloud and Outposts. Kong Mesh sits at the heart of how our services talk to each other - securing traffic, managing policies, handling retries and timeouts. You'll bring expertise on Kubernetes and mesh and help us operate these platforms with confidence and

Full-time

USD 159,000.00 - 208,950.00 per year

Search all similar jobs

Remote jobs at Perplexity AI