Roles & Responsibilities
Own end-to-end product execution across knowledge ingestion, retrieval quality, grounded answer generation, enterprise chat experience, guardrails, observability, and agentic use case rollout.
Work closely with end users to understand pain points, validate use cases, and ensure the product improves productivity and trust.
Partner with engineering teams to clarify requirements, resolve functional gaps, and drive delivery against defined milestones.
Own product testing and business validation before every release, including scenario coverage, acceptance criteria validation, regression checks, and readiness signoff.
Own quality of the RAG pipeline by partnering with technical teams on ingestion readiness, chunking/indexing implications, retrieval tuning, grounding, citation behavior, and answer-quality improvements.
Define and own the evaluation framework, including metrics for retrieval relevance, answer correctness, faithfulness, hallucination rate, citation accuracy, fallback handling, and latency — ensuring every test run is measured against established baselines.
Maintain product-level feedback loops using golden datasets, user feedback, UAT findings, and post-release learning to continuously improve quality.
Drive observability requirements so the team can monitor end-to-end query behavior, retrieval outcomes, response quality, latency, errors, drift, and user feedback signals.
Participate in product decisions related to cloud-based GenAI architecture, including ingestion pipeline optimizations and model usage / cost-quality balance.
Support rollout of agentic automation use cases for lower-complexity workflows, while ensuring proper safety controls, approval flows, and auditability are in place.
Skills
GenAI / RAG Skills
Experience building and shipping enterprise-grade GenAI products, especially RAG-based assistants or enterprise knowledge assistants.
Understanding of the end-to-end RAG lifecycle, including data ingestion, document parsing, chunking, metadata enrichment, embeddings, indexing, retrieval, grounding, citation, and response synthesis.
Familiarity with cloud-based GenAI architecture, including managed LLM services, vector/search indexes, object storage, workflow orchestration, serverless compute, containerized services, and enterprise integration patterns.
Ability to engage with engineering on retrieval tuning, top-k strategy, hybrid search, re-ranking, query understanding, prompt structure, grounding, hallucination control, and answer-quality tradeoffs.
Understanding of guardrails and responsible AI for enterprise assistants, including content safety, PII handling, topic boundaries, confidence signaling, access control, and audit trails.
Understanding of observability and monitoring for LLM products, including trace-level visibility, query lifecycle analysis, latency, token usage, drift, alerting, rollback, and user feedback instrumentation.
Working knowledge of LLM capabilities and limitations, model tradeoffs, prompt architecture, and prompt vulnerabilities such as injection and jailbreak risks.
Exposure to agentic AI concepts such as tool use, function calling, multi-step reasoning, planning, human-in-the-loop approvals, and safe autonomous execution.
Product Management Skills
Strong execution and delivery management skills, with the ability to drive a defined roadmap across multiple technical teams and dependencies.
Experience writing clear requirements, acceptance criteria, and release scope for technically complex products.
Experience owning product validation and testing before release, with clear quality bars and definition of done.
Excellent stakeholder management skills across business users, end-user teams, engineering, platform, security, and leadership.
Strong communication skills, including ability to explain AI quality, trust, retrieval issues, and guardrails in business-friendly language.
Experience running UAT, gathering nuanced user feedback, and converting findings into measurable product improvements.
Never miss an opportunity! Create an alert based on the job you applied for.
