Qualifications:
Leadership & Core Experience
Proven Technical Leadership: 8+ years of experience architecting and operating production backend or distributed systems, including at least 2+ years acting as a Technical Lead.
Educational Foundation: Bachelor’s degree in Computer Science, a related technical field, or equivalent practical experience.
AI & Architecture Expertise
Production AI Tracking: Hands-on experience successfully deploying LLM-powered applications or agents to production, with a strong understanding of handling edge cases, failures, and mitigation.
Framework & Tooling Engineering: Advanced Python proficiency combined with a track record of designing robust frameworks, developer SDKs, or platform APIs tailored for other engineering teams.
Foundational Engineering & Communication
Distributed Systems Mastery: Deep understanding of distributed system fundamentals, including API design, asynchronoevent-driven processing, data streaming, caching strategies, rate limiting, and fault-tolerant architectures.
Technical Communication: Excellent written communication skills, with proven ability to author comprehensive system design documents that articulate technical trade-offs clearly to both engineering teams and executive leadership.
Technical Capabilities & Domain Expertise:
We are looking for a well-rounded technical leader with broad exposure across these domains and deep expertise in a few key areas:
Agent Orchestration & Interoperability
Agent Development: Architecting tool/function calling, multi-step and multi-agent orchestration, task delegation, durable execution, and human-in-the-loop approval workflows.
Agent Interoperability: Designing open protocols to expose agents and tools seamlessly to other applications and external agents.
AI User Experiences: Engineering streaming agent outputs to web clients and building generative or dynamic user interfaces.
Context Engineering & Lifecycle Management
Context Engineering: Implementing Retrieval-Augmented Generation (RAG), vector/hybrid search, short- and long-term memory systems, automatic summarization, and routing across extensive tool catalogs.
LLM Evaluation & Observability: Building evaluation datasets and rubrics, leveraging LLM-as-a-judge, running regression testing in CI, and establishing distributed tracing, logging, metrics, SLOs, and online production evaluation.
Enterprise Platform, Security & Governance
LLM Infrastructure: Managing cross-provider model routing, token consumption, cost optimization, response caching, and latency reduction.
AI Safety & Governance: Deploying prompt-injection defenses, PII detection/redaction, guardrails, policy engines, and sandboxed code execution environments.
Core & Multi-Tenant Security: Designing authentication/authorization, service-to-service identity, least-privilege access controls, strict tenant isolation, and detailed audit logging.
Cloud-Native Infrastructure: Operating containers, container orchestration, robust CI/CD pipelines, and infrastructure as code (IaC).
Core Tech Stack & Toolkit
Expertise in Python is required. For the rest of the stack, we look for hands-on exposure to most of these technologies:
Languages & Backend Architecture
Languages: Expert-level Python; solid working knowledge of SQL. Java experience is a valuable asset.
Backend APIs & Streaming: High proficiency in FastAPI, asyncio, and Pydantic; practical experience with REST, gRPC, Server-Sent Events (SSE), WebSockets, and event streaming via Kafka or similar.
AI, Agentic Frameworks & Protocols
Agent Frameworks & Protocols: Hands-on experience with LangGraph and LangChain, or comparable frameworks (e.g., OpenAI Agents SDK, Google ADK, Pydantic AI) along with agent protocols like MCP, A2A, and A2UI.
Models, Providers & Gateways: Interfacing with enterprise APIs (OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Azure OpenAI), serving open-weight models via vLLM, and managing LLM gateways like LiteLLM.
Data, Evaluation & Observability
Data & Retrieval Engines: PostgreSQL with pgvector; vector and search engines (OpenSearch, Elasticsearch, Qdrant); Redis; and embedding/reranking models.
Evaluation & Telemetry: OpenTelemetry (including GenAI semantic conventions); LLM tracing/eval platforms (LangSmith, Ragas, DeepEval); and standard metrics stacks (Prometheus, Grafana).
Security, Infrastructure & Frontend
Security & Governance Tools: OAuth 2.0, OIDC, JWT, and workload identities (SPIFFE/SPIRE); policy-as-code (Open Policy Agent, Cedar); secrets management (HashiCorp Vault); and guardrail frameworks (NeMo Guardrails, Llama Guard, Presidio).
Cloud-Native & Sandboxing: Docker, Kubernetes, Helm, and Terraform; experience with at least one major cloud provider (AWS, Azure, Google Cloud Platform); GitHub Actions or Jenkins; and secure sandbox runtimes (gVisor, Firecracker).
Frontend Engineering: React; streaming chat interfaces; and generative UI frameworks or specifications (Vercel AI SDK, A2UI, MCP-UI).
Preferred Qualifications:
Platform Scale & Adoption: Proven experience building internal platforms, developer SDKs, or frameworks that were successfully adopted across multiple engineering teams.
Enterprise AI Experience: Track record of deploying, running, and managing production AI systems tailored for external enterprise customers.
Open Source & Ecosystem Impact: Active contributions to open-source agentic frameworks or emerging agent protocols.
