AI Tester

AI Tester
Required Skills:
Needs to be proficient in Selenium Automation Testing
Candidates that have interviewed / screened have been proficient in manual testing but do not have the necessary experience with automation testing using selenium - THIS IS REQUIRED
AI Testing exp - needs to be proficient in validating GenAI applications
Candidates have been not able to speak to validating / automating GenAI applications - this is also required

Overall - they are looking for a strong Java coder with strong selenium automating testing with experience in AI testing, validating the GenAI apps.

Experience:
* 6+ years of experience in Software Functional testing with Automation (Selenium) knowledge

Required
Testing Concepts:
* Responsible for system integration testing of newly developed or enhanced applications
* Requires in-depth knowledge of the software implementation lifecycle
* Possesses good Testing Process knowledge and a detailed understanding on AGILE methodology
* Works with project teams during early stages to establish plans, standards, and procedures that will add value to the testing effort and satisfy the constraints of the project
* Ensures on-time delivery of work including monitoring of external and internal dependencies, tracking of progress, and monitoring of project milestones accomplishments by: o Recording work status information and generating status reports
* Tracking and reporting actual versus planned completion
* Ensuring all activity is accurately recorded and reported
* Drives the education of Business and Development teams on the testing model, methodology, and processes, particularly end-to-end testing, and its role in quality assurance
* Demonstrates Leadership characteristics while handling Product, Development, EAS, and other stakeholders
* Demonstrates proficiencies in Project and Quality management, Issue management, and Communication
* Assists Testing Manager in reviewing Portfolio Management and Resource Management forecast reports, to optimize team's supply/demand planning, maximizing resource utilization
* Monitors adherence to quality standards during the testing of production applications, identifies areas of strengths and weakness
* Identifies process improvement opportunities and communicates to the project coordinator
* Experience with Jira and ALM

Domain Knowledge:
* Ability to works closely with business and development subject matter experts, to continually improve depth and breadth of knowledge for the assigned applications/systems

Automation:
* Experience in writing, and executing application tests using industry standard automated testing tools like Selenium

AI Specific Testing:
* Hands on experience testing LLMs and RAG architectures using vector databases or embeddings to validate retrieval accuracy in RAG based models.
* Implement automated testing frameworks for gen AI applications such as Deep eval, RAGAs etc for evaluating and testing LLMs
* Strong knowledge of Python and experience with AI testing tools and libraries (e.g, LangChain, Pytest, Pytorch)
* Develop custom validation scripts for Gen AI model output evaluation including accuracy, relevancy, faithfulness and performance benchmarks
* Test prompt engineering techniques to optimize responses
* Familiarity with Snowflake and executing SQL queries to validate database integrity and application responses
Comments for Suppliers: