Overwatch Guardrails Engineer
Concord, CA (Onsite)
6 months
The Overwatch Guardrails Engineer is responsible for ensuring enterprise AI solutions are safe, secure, compliant, and production ready. This role designs and automates guardrail validation, adversarial testing, and governance controls for AI models and agents, while generating auditable evidence to support deployment approvals and regulatory compliance.
· Design, implement, and validate AI guardrails for LLMs, RAG, and agent-based solutions.
· Automate adversarial and red-team testing to identify security, safety, and policy vulnerabilities.
· Develop test harnesses and evaluation frameworks to assess model and agent readiness.
· Support model and agent enablement gates through risk, compliance, and control validation.
· Generate repeatable, auditable evidence packages for governance and approval processes.
· Integrate safety, security, and governance controls into CI/CD and AI delivery pipelines.
· Partner with Responsible AI, Security, Risk, Compliance, and Engineering teams to ensure regulatory adherence.
· Continuously enhance testing frameworks, controls, and monitoring based on emerging threats and industry standards.
· Responsible AI and AI governance frameworks
· Adversarial testing and AI red teaming
· Content moderation and policy guardrails
· AI security and vulnerability testing
· Model risk management and regulatory controls
· Test automation and evaluation frameworks
· Python development and scripting
· Evidence generation and compliance automation
· Experience delivering solutions in regulated environments (Banking, Financial Services, Healthcare, etc.)
· Experience with LLMs, GenAI, AI agents, and RAG architectures.
· Familiarity with governance frameworks such as NIST AI RMF, ISO 42001, or similar standards.
· Knowledge of CI/CD, cloud platforms, and enterprise AI deployment practices.
A successful candidate combines AI safety, security, governance, and engineering expertise to build scalable guardrails that enable rapid, compliant, and trustworthy AI adoption across the enterprise.
Never miss an opportunity! Create an alert based on the job you applied for.
