Design and run intelligent test strategies for LLM and agent-based systems, combining classic QA and test automation skills with a deep understanding of agentic automation,and conversational AI,
Responsibilities
Design and execute end-to-end test plans for UX / Uber orchestrator/ Sub Agents . Build and maintain agentic AI test suites validating autonomous . Develop test data and synthetic scenarios for edge cases, ambiguous queries, and multilingual requests.
Evaluate agent behavior through reasoning traces, tool-use decisions, and memory usage.
Implement and improve automated testing frameworks leveraging AI for conversational flows, APIs, and integrations.
Collaborate with Product management / cross-functional teams to refine requirements and acceptance criteria.
Document defects, test results, and insights clearly for stakeholders.
Skill Set
Minimum 5 years of experience in software QA / test engineering. Strong understanding of testing principles: functional, regression, integration, performance.
Hands-on experience testing AI or ML systems such as LLM chatbots or recommendation engines.
Familiarity with key concepts of multi-agentic systems, agentic frameworks (e.g., LangChain), AWS AgentCore.
Scripting skills in Python and experience with REST API testing
Hands on experience with AI and Machine Learning (ML) techniques to enhance test automation, such as generating test cases, predicting potential defects, or updating test code for user interface changes.
Hands on experience with test automation frameworks and tools like Selenium, Playwright or Cucumber using scripting languages like Python, Java, or JavaScript and leveraging AI to improve productivity .
Excellent analytical, documentation, and communication skills.
Collaborate with cross-functional teams for requirements clarifications and acceptance criteria.
Document defects, test results, and insights clearly for stakeholders.
Never miss an opportunity! Create an alert based on the job you applied for.
