AI Agentic Tester

Role: AI Agentic Tester

Denver, CO (Zip Code-80221)- REMOTE

Duration: 6 months

Essential Skills: AI Agentic Tester

Job Description:

Functional Testing

• Validate AI agent workflows against business requirements.
• Test end-to-end AI use cases, including multi-agent interactions.
• Verify task completion accuracy and output quality.
• Test agent orchestration, tool calling, and workflow execution.

AI Model Validation

• Evaluate LLM responses for accuracy, relevance, and consistency.
• Test prompt engineering effectiveness and response quality.
• Validate Retrieval-Augmented Generation (RAG) outputs.
• Measure hallucination rates and response reliability.

Required Skills

• 3-8 years of QA/Test Engineering experience.
• Experience testing Generative AI, LLM, or AI-based applications.
• Knowledge of AI Agents, Agentic AI, RAG, and prompt engineering.
• Hands-on experience with API testing (Postman, Swagger, REST APIs).
• Experience with automation frameworks (Selenium, Playwright, PyTest).
• Proficiency in Python.
• Understanding of AI evaluation metrics and testing methodologies
• Knowledge of Azure OpenAI, AWS Bedrock, Gemini, or similar AI platforms