Engineering Manager (AI Inference)

San Francisco, CA, US • Posted 3 hours ago • Updated 3 hours ago
Full Time
On-site
USD $300,000.00 - 485,000.00 per year
Fitment

Dice Job Match Score™

👤 Reviewing your profile...

Job Details

Skills

  • Art
  • Technical Direction
  • Python
  • Rust
  • C++
  • Sonar
  • Scratch
  • Legacy Systems
  • Spectrum
  • Roadmaps
  • Artificial Intelligence
  • Incident Management
  • Mentorship
  • Operational Excellence
  • IT Management
  • Management
  • Machine Learning (ML)
  • PyTorch
  • TensorFlow
  • Fusion
  • GPU
  • Performance Analysis
  • Real-time
  • Technical Communication
  • Collaboration
  • CUDA
  • Training
  • Kubernetes
  • Orchestration
  • Optimization
  • Research

Summary

About the Role

We are looking for an Inference Engineering Manager to lead our AI Inference team. This is a unique opportunity to build and scale the infrastructure that powers Perplexity's products and APIs, serving millions of users with state-of-the-art AI capabilities.

You will own the technical direction and execution of our inference systems while building and leading a world-class team of inference engineers. Our current stack includes Python, PyTorch, Rust, C++, and Kubernetes. You will help architect and scale the large-scale deployment of machine learning models behind Perplexity's Comet, Sonar, Search, Deep Research products.
Why Perplexity?
  • Build SOTA systems that are the fastest in the industry with cutting-edge technology
  • High-impact work on a smaller team with significant ownership and autonomy
  • Opportunity to build 0-to-1 infrastructure from scratch rather than maintaining legacy systems
  • Work on the full spectrum: reducing cost, scaling traffic, and pushing the boundaries of inference
  • Direct influence on technical roadmap and team culture at a rapidly growing company
Responsibilities
  • Lead and grow a high-performing team of AI inference engineers
  • Develop APIs for AI inference used by both internal and external customers
  • Architect and scale our inference infrastructure for reliability and efficiency
  • Benchmark and eliminate bottlenecks throughout our inference stack
  • Drive large sparse/MoE model inference at rack scale, including sharding strategies for massive models
  • Push the frontier with building inference systems to support sparse attention, disaggregated pre-fill/decoding serving, etc.
  • Improve the reliability and observability of our systems and lead incident response
  • Own technical decisions around batching, throughput, latency, and GPU utilization
  • Partner with ML research teams on model optimization and deployment
  • Recruit, mentor, and develop engineering talent
  • Establish team processes, engineering standards, and operational excellence
Qualifications
  • 5+ years of engineering experience with 2+ years in a technical leadership or management role
  • Deep experience with ML systems and inference frameworks (PyTorch, TensorFlow, ONNX, TensorRT, vLLM)
  • Strong understanding of LLM architecture: Multi-Head Attention, Multi/Grouped-Query Attention, and common layers
  • Experience with inference optimizations: batching, quantization, kernel fusion, FlashAttention
  • Familiarity with GPU characteristics, roofline models, and performance analysis
  • Experience deploying reliable, distributed, real-time systems at scale
  • Track record of building and leading high-performing engineering teams
  • Experience with parallelism strategies: tensor parallelism, pipeline parallelism, expert parallelism
  • Strong technical communication and cross-functional collaboration skills
Nice to Have
  • Experience with CUDA, Triton, or custom kernel development
  • Background in training infrastructure and RL workloads
  • Experience with Kubernetes and container orchestration at scale
  • Published work or contributions to inference optimization research
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 91140125
  • Position Id: 6845d2bc522773e5cf0fc4ddff08a90
  • Posted 3 hours ago
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

San Francisco, California

Today

Full-time

USD 167,200.00 - 209,000.00 per year

San Francisco, California

Today

Full-time

San Francisco, California

Today

Full-time

USD 300,000.00 - 485,000.00 per year

Remote or San Francisco, California

Today

Full-time

USD 177,185.00 - 364,795.00 per year

Search all similar jobs