Role: AI Agent Developer & Tester
Location: 100% remote role
Duration: 8-month contract
Job Description:
Our client is seeking an AI/Agent Developer & Tester to support the development, testing, and deployment of AI agent workflows within their platform. This role will be responsible for building and validating LangGraph-based agents, developing automated test coverage, and ensuring reliability across development and production environments.
The ideal candidate will have strong experience with Python, LangGraph, and modern testing frameworks. Responsibilities include creating node-level unit tests, graph-level integration tests, regression testing, and production monitoring for AI-powered applications. This individual will also manage end-to-end UI testing using Playwright and support cloud-based infrastructure that powers the AI platform.
This role requires a quality-focused engineer who can work independently, build scalable testing practices, and help deliver stable, production-ready AI solutions.
Required Skills:
Experience with LangGraph and LangChain
Strong Python development experience
Experience with pytest and pytest-asyncio
Experience building unit tests for AI agents and workflow nodes
Experience creating end-to-end agent and graph integration tests
Knowledge of LLM mocking and test isolation strategies
Experience managing non-deterministic AI behavior through controlled testing methods
Experience designing and maintaining regression test suites
Playwright end-to-end testing experience
Test coverage reporting and analysis experience
Experience with JSON schema validation and structured outputs
Production monitoring and observability experience
Cloud application development and support experience (Google Cloud Platform preferred)
Strong troubleshooting and problem-solving skills
BonSoft Skills:
Experience with LangSmith or RAGAS
Prompt versioning experience
Adversarial or red-team testing experience
Token usage and cost monitoring experience
Experience with document extraction or OCR workflows
Experience in financial services or other regulated industries
Experience with Google Cloud Platform and Cloud Run
Terraform experience
Azure DevOps pipeline experience
Strong communication and collaboration skills