SDET/QA (Python/AI)-Fully Remote

Overview

Remote
$DOE
Accepts corp to corp applications
Contract - W2
Contract - Independent
Contract - 12

Skills

.

Job Details

Remote

SDET/QA (Python/AI); LLM, Machine Learning

Title: Software Development Engineer in Test

Key Skills: Python, any Scripting (Javascript, Shell, Perl), AI or Machine Learning product experience

Job Description:

TWO SDET QAs with exp in python and scripting. Experience with AI tech like Lang Graph is a plus

Resources will be responsible for testing Scout OpenAI and LLM, developing a virtual recruiter, and automating tasks. The solution will be built as an SDK, allowing integration with third-party ATS platforms while maintaining its current functionality within Dradis

Perform usability testing by interacting with the system through natural language commands (e.g., "Show me top applicants") and assessing its responses.

Analyze LLM behavior, identifying areas for improvement and providing actionable feedback.

Design and implement a test framework tailored for evaluating LLM performance, requiring an innovative approach beyond standard models.

Conduct backend automated testing or frontend manual testing, with no requirement for frontend automation scripts.

Work independently to log and track bugs, proactively identifying and implementing automation opportunities where applicable.

Evaluate model interactions, testing search relevance by querying top applicants and refining prompts for improved results.

Validate workflow functionality, leveraging LangGraph to assess system effectiveness and ensure seamless operations.

The resource will start writing a test framework that can test and evaluate LLMs (this is not a standard model)

Industry experience probably won't be found since this is so new

Either FE manual testing or backend automated testing

Independent workers

No FE automation scripts will be asked for

Log bugs

Figure out if automation is an option and start doing the work

Automate routine tasks

Required Skills

Python and Scripting Experience: Proficiency in writing Python scripts is essential for executing test scripts. Prior experience testing an AI chat agent/assistant or tool using Python scripts is very helpful.

Experience in a manual testing role, identifying and reporting bugs with the ability to automate routine and repeated tasks to optimize future testing efforts

Cross-functional Collaboration:

Ability to work effectively with cross-functional teams/Managers

Nice to have:

Experience with AI Technologies like LangGraph: While not mandatory, experience with AI technologies such as LangGraph is a plus.

No prior LLM experience is required but is a plus

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.