AI Test Engineer

12+ years of experience

Define the end-to-end test strategy for non-deterministic systems: golden datasets, LLM-as-judge evals, regression suites, and human review loops.

Build automated evaluation harnesses measuring accuracy, groundedness, toxicity, bias, latency, and multimodal quality (speech recognition accuracy, image understanding, voice naturalness).

Conduct adversarial and red-team testing: jailbreaks, prompt injection, data leakage, and edge cases across languages, accents, and accessibility needs.

Automate functional, performance, and load testing across browsers, iOS, and Android, integrated as quality gates in CI/CD.

Test Harness, DeepEvals, Playwright, Mabl, QAwolf, and other AI testing tool experience is required.

Track defect trends and release-readiness metrics, and report quality risks clearly to engineering and the project manager.