Job Title: SRE Engineer
Location: US(Remote)
Duration: Long term contract
Visa: USC
The AI Platform and Services team provides the shared runtime and developer experience that power AI capabilities. We operate multi-tenant inference, VectorDB/RAG, LangSmith and evaluation integrations, MCP services, and the platform controls product teams need to build secure, observable, reliable, and cost-efficient AI experiences.
You will be a hands-on engineer improving production reliability and automating repetitive operational work. This role is well suited to an engineer who enjoys distributed systems, Kubernetes, observability, incident reduction, performance optimization, and cloud cost accountability.
Your Impact
- Build automation that helps the platform progress toward a 99.9% service-availability target.
- Improve telemetry and alerting for latency, token throughput, queueing, KV-cache pressure, preemption, GPU utilization, availability, and cost.
- Establish repeatable model benchmarks, stress tests, capacity baselines, and SLO-based readiness checks.
- Improve autoscaling, model cold-start behavior, canary deployment, promotion, rollback, and incident remediation.
- Automate production operations, including health and security checks, upgrades, backups, configuration validation, and runbook-driven recovery.
- Improve AWS and GPU cost attribution, anomaly detection, capacity reporting, and COGS analysis for internal cloud accounts.
- Contribute to centralized identity, authorization, policy, and auditability in AgentGateway.
- Support approved model onboarding and hosting work, including compatibility validation, provenance, deployment, benchmarking, and operational readiness.
- Participate in production support, incident response, root-cause analysis, and follow-up automation.
- Create maintainable code, tests, dashboards, documentation, and operational runbooks.
Minimum Qualifications
- Bachelor’s degree in computer science, engineering, or equivalent practical experience.
- 4+ years of hands-on experience building or operating backend, cloud, infrastructure, SRE, or distributed systems in production.
- Proficiency in at least one modern programming language such as Python, Go, or Java.
- Experience with containers, Kubernetes, CI/CD, service discovery, configuration management, and production microservices.
- Experience creating dashboards, alerts, SLOs, runbooks, and operational automation.
- Demonstrated production ownership, including debugging, incident response, root-cause analysis, and reliability improvements.
- Strong communication and collaboration skills across engineering and product teams.
Preferred Qualifications
- Experience operating LLM or deep-learning inference using vLLM, Ray, Triton Inference Server, TensorRT-LLM, SGLang, or similar systems.
- Understanding of LLM-serving performance, including batching, chunked prefill, caching/KV cache, quantization or mixed precision, and GPU utilization.
- Experience with Prometheus, Grafana, OpenTelemetry, Splunk Observability, or comparable monitoring platforms.
- Experience with AWS, Azure, or Google Cloud Platform infrastructure, networking, IAM, Kubernetes, and GPU workloads.
- Experience with autoscaling, capacity planning, canary releases, rollback automation, and multi-region or high-availability systems.
- Experience with FinOps, cloud cost allocation, cost anomaly detection, or service-level COGS reporting.
- Familiarity with authentication, authorization, policy enforcement, audit logging, or API gateway patterns.
What Success Looks Like
Reliability, deployment, rollback, incident-response, benchmarking, and cost-reporting workflows require materially less manual work.
Platform services have clear SLOs, actionable dashboards and alerts, tested recovery procedures, and repeatable capacity baselines.
Model onboarding and hosting use automated compatibility, benchmarking, provenance, canary, and rollback workflows.
Operational improvements measurably reduce incidents, recovery time, manual toil, and infrastructure cost.
