QA Automation Engineer

We are seeking an experienced QA Automation Engineer with strong hands-on experience in Python-based test automation and emerging Generative AI / Agentic AI technologies, particularly LangChain and LangGraph.

The ideal candidate will be responsible for designing, developing, and maintaining automated testing frameworks and validating AI-powered applications, workflows, APIs, and agent-based solutions. The candidate should have strong programming and automation skills along with an understanding of LLMs, AI agents, and AI application testing.

Key Responsibilities

• Design, develop, and maintain automated test frameworks using Python.
• Develop and execute automated functional, regression, integration, API, and end-to-end tests.
• Build reusable automation scripts and testing utilities using Python.
• Test and validate Generative AI, LLM, and Agentic AI applications.
• Work with LangChain and LangGraph to understand and validate AI workflows and agent-based applications.
• Develop test scenarios for LLM applications, AI agents, chains, tools, and workflows.
• Validate AI application responses for accuracy, consistency, reliability, and expected behavior.
• Perform API testing and integration testing for AI and backend services.
• Identify, document, troubleshoot, and track defects throughout the software development lifecycle.
• Integrate automated tests into CI/CD pipelines.
• Collaborate closely with Developers, QA Engineers, Product Owners, and AI/ML teams.
• Analyze test results, identify failures, and perform root-cause analysis.
• Create and maintain comprehensive test cases, test scripts, automation frameworks, and test documentation.
• Continuously improve test coverage, automation efficiency, and overall product quality.
• Participate in Agile/Scrum ceremonies and contribute to continuous quality improvement.

Required Skills

• Strong hands-on experience in Python programming.
• Strong experience in QA Automation / Test Automation.
• Experience developing automation frameworks using Python.
• Strong knowledge of Selenium, Playwright, PyTest, or similar automation tools.
• Experience with API testing using tools/frameworks such as REST API, Postman, or Python-based API testing.
• Hands-on or strong working knowledge of LangChain and/or LangGraph.
• Understanding of Generative AI, LLMs, AI Agents, and Agentic AI workflows.
• Experience testing AI-powered applications and complex workflows.
• Strong understanding of functional, regression, integration, API, and end-to-end testing.
• Experience with Git and CI/CD pipelines.
• Strong debugging, analytical, and problem-solving skills.
• Good understanding of Agile/Scrum methodologies.

Preferred Skills

• Experience with OpenAI, Azure OpenAI, Anthropic Claude, or other LLM platforms.
• Experience with LangChain, LangGraph, or other AI agent frameworks.
• Knowledge of prompt engineering and LLM evaluation.
• Experience validating AI/LLM responses and AI agent workflows.
• Familiarity with RAG (Retrieval-Augmented Generation) applications.
• Experience with vector databases such as Pinecone, FAISS, Chroma, or Weaviate.
• Knowledge of cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Experience with Docker and Kubernetes is a plus.
• Experience working in CI/CD environments such as Jenkins, GitHub Actions, GitLab CI, or Azure DevOps.