Senior AI/ML QA Engineer - Remote

Position: Senior AI/ML QA Engineer Remote Contract:W2/1099

Key Responsibilities:

1. QA Strategy & Leadership

• Lead the development of comprehensive testing strategies for AI/ML systems.
• Define test plans, test cases, and acceptance criteria for machine learning models, APIs, and data pipelines.
• Collaborate with data scientists, ML engineers, and software teams to ensure quality across the ML lifecycle.

1. Model Validation & Evaluation

• Validate model performance, accuracy, precision, recall, F1 score, ROC-AUC, and other key metrics.
• Detect and address issues related to model bias, overfitting, drift, and instability.
• Evaluate models in different environments (e.g., pre-prod vs prod) and datasets.

1. General :

• Own and drive QA processes for key product areas, requiring minimal oversight or guidance.
• Design, develop, and execute comprehensive test cases for new features and core platform functionality.
• Conduct regression testing to ensure new updates do not impact existing functionality.
• Lead UI/UX testing efforts to validate user-facing flows and interaction models.
• Collaborate with engineering and product teams to define testing requirements and quality benchmarks.
• Analyze and reproduce bugs reported by offshore teams; write detailed bug reports and track resolution.
• Work closely with developers to validate fixes and confirm coverage.
• Understand and test around LLM-specific features, prompt flows, and backend behavior.
• Verify and troubleshoot API keys, integrations, and dev tools used within the platform.

Qualifications:

• 5+ years of hands-on QA experience, ideally with web-based or developer-focused platforms.
• Strong understanding of software testing methodologies, automation strategies (even if this role is primarily manual), and agile workflows.
• Proven experience working with LLM or AI/ML-based systems understanding how language models operate and are integrated into applications.
• Excellent communication skills to drive bug resolution and coordinate with offshore QA teams.
• Familiarity with developer environments, including handling API keys, SDKs, CLI tools, and relevant documentation.
• Demonstrated ability to think critically and creatively to identify edge cases and usability issues.
• Experience with bug tracking and test management tools such as Jira, TestRail, etc.

Preferred:

• Prior experience testing AI tools, IDEs, or platforms for developers.
• Background in functional and exploratory testing for user-facing features powered by LLMs.
• Familiarity with Chrome DevTools or browser debugging utilities.
• Basic scripting or automation knowledge is a plus (Python, JavaScript, etc.).