We have an excellent opportunity for a Mid-Level AI Software Test Engineer / SDET with strong experience in software testing, automation, and AI/ML application validation.
Position: Mid-Level AI Software Test Engineer / SDET
Location: Ann Arbor, MI
Work Arrangement: Onsite
Contract: 18 Months
Schedule: First 6 months – 100% onsite; after 6 months – 4 days onsite and 1 day remote
Experience: 3–6 years in Software Quality Engineering, QA Automation, or SDET
Position Overview
We are seeking an AI Software Test Engineer to design, automate, and execute testing strategies for AI-powered applications, machine learning systems, AI agents, copilots, and Generative AI solutions. The ideal candidate will combine strong software quality engineering skills with experience or interest in AI technologies such as LLMs, RAG, AI agents, MCP integrations, and machine learning models.
Key Responsibilities
- Develop comprehensive testing strategies for AI applications, platforms, and services.
- Validate AI outputs for accuracy, consistency, reliability, and safety.
- Design and execute functional, integration, end-to-end, regression, and performance testing.
- Create test cases for prompt-driven, agentic, and retrieval-based AI workflows.
- Validate AI guardrails, permissions, business rules, and governance controls.
- Perform adversarial, negative, edge-case, and hallucination testing.
- Build and maintain automated testing frameworks and AI evaluation pipelines.
- Integrate AI testing into CI/CD pipelines and implement automated quality scoring.
- Test AI agents, APIs, workflows, MCP integrations, and enterprise system integrations.
- Perform load, stress, scalability, reliability, and resiliency testing.
- Collaborate with software engineers, AI engineers, architects, product owners, and security teams.
- Support production readiness, defect triage, quality standards, and best practices.
Required Qualifications
- Bachelor's degree in Computer Science, Software Engineering, Information Systems, or related field.
- 3–6 years of software testing, QA automation, or quality engineering experience.
- Experience developing automated test solutions using Python, Java, JavaScript/TypeScript, or C#.
- Strong API testing and automation experience.
- Strong understanding of test automation, SDLC, Agile, and CI/CD.
- Experience testing distributed systems, web applications, and APIs.
Preferred Qualifications
- Experience testing Generative AI, LLM-based applications, AI agents, RAG applications, or MCP integrations.
- Familiarity with OpenAI, Claude, Gemini, or Azure OpenAI.
- Experience building AI evaluation or benchmarking frameworks.
- Cloud experience with Azure, AWS, or Google Cloud Platform.
- Knowledge of responsible AI, AI governance, risk, privacy, and compliance.
Technical Skills
AI Testing: Prompt Testing, Response Evaluation, Hallucination Detection, Agent Workflow Validation, RAG Validation, AI Safety Testing, AI Regression Testing, AI Benchmarking.
Quality Engineering: Test Automation, API Testing, Integration Testing, Performance Testing, Load Testing, Security Testing, Defect Analysis, Root Cause Investigation.
Tools: Selenium, Playwright, Postman, JUnit, PyTest, GitHub Actions, Jenkins, Docker, Kubernetes, Azure, AWS, Google Cloud Platform.
Equivalent Job Titles
- AI Software Test Engineer
- AI Quality Engineer
- Generative AI Test Engineer
- AI Automation Engineer
- AI Validation Engineer
- SDET – AI
- Software Engineer in Test – AI
The ideal candidate should be capable of independently owning AI testing and automation initiatives while working closely with AI developers, architects, and product teams to deliver reliable, secure, and production-ready AI solutions.
Never miss an opportunity! Create an alert based on the job you applied for.
