Genesis10 is currently seeking a QA and Agentic AI Validation Engineer for a contract position with a Global Financial Institution located in Pennington, NJ. This is a 12+ month contract opportunity. This role bridges the gap between traditional quality engineering and autonomous AI systems. The successful candidate will be responsible for architecting autonomous, agent-driven testing frameworks and building rigorous evaluation pipelines to validate the reliability, safety, and accuracy of AI agents. You will ensure our non-deterministic AI workflows behave predictably, adhere to strict security boundaries, and deliver measurable business value.
Responsibilities: - Design and execute evaluation frameworks for multi-step autonomous workflows, testing reasoning paths, tool-calling accuracy, state management, and task completion
- Build automated grading pipelines and calibrate model-based judges against human baseline ratings to measure accuracy, hallucination rates, and consistency
- Validate Retrieval-Augmented Generation (RAG) pipelines for retrieval precision, groundedness, citation accuracy, and adherence to user data-access permissions
- Probe agents for prompt injection vulnerabilities, unauthorized tool execution, data leakage, infinite loops, and failure-recovery behavior (e.g., API timeouts or malformed tool outputs)
- Deploy and supervise AI testing agents to generate test scenarios from acceptance criteria, prioritize regression suites based on code risk, and execute browser/API journeys
- Define strict permission rules, execution hooks, and sandbox boundaries for testing agents-reviewing agent-authored assertions to prevent false positives or "assertion loosening"
- Integrate deterministic automated tests and probabilistic AI evals into CI/CD pipelines, owning release-readiness evidence, latency benchmarks, and token-cost tracking
Requirements: - 5+ years in Software Quality Engineering / SDET roles, including 1-2+ years of hands-on experience validating LLM applications, RAG systems, or autonomous AI agents
- Strong proficiency in Python (or TypeScript) along with modern test automation frameworks (e.g., Playwright, Pytest, Cypress) and REST/GraphQL API testing
- Practical experience with agent orchestration and evaluation tools (e.g., LangChain/LangGraph, CrewAI, AutoGen, Dify, DeepEval, Ragas, Promptfoo, or Braintrust)
- Ability to inspect execution traces, span logs, and tool-call payloads (using tools like LangSmith, Arize Phoenix, or OpenTelemetry) to diagnose non-deterministic failures
- Working knowledge of OWASP Top 10 for LLMs, PII/data-privacy compliance, and guardrail enforcement
Pay range: $55.06 - $63.06 per hour
Only candidates available and ready to work directly as Genesis10 employees will be considered for this position. If you have the described qualifications and are interested in this exciting opportunity, please apply! Ranked a Top Staffing Firm in the U.S. by Staffing Industry Analysts for six consecutive years, Genesis10 puts thousands of consultants and employees to work across the United States every year in contract, contract-for-hire, and permanent placement roles. With more than 300 active clients, Genesis10 provides access to many of the Fortune 100 firms and a variety of mid-market organizations across the full spectrum of industry verticals.
For contract roles, Genesis10 offers the benefits listed below. If this is a perm-placement opportunity, our recruiter can talk you through the unique benefits offered for that particular client. Benefits of Working with Genesis10: - Access to hundreds of clients, most who have been working with Genesis10 for 5-20+ years.
- The opportunity to have a career-home in Genesis10; many of our consultants have been working exclusively with Genesis10 for years.
- Access to an experienced, caring recruiting team (more than 7 years of experience, on average.)
- Behavioral Health Platform
- Medical, Dental, Vision
- Health Savings Account
- Voluntary Hospital Indemnity (Critical Illness & Accident)
- Voluntary Term Life Insurance
- 401K
- Sick Pay (for applicable states/municipalities)
- Commuter Benefits (Dallas, NYC, SF, and Illinois)
For multiple years running, Genesis10 has been recognized as a Top Staffing Firm in the U.S., as a Best Company for Work-Life Balance, as a Best Company for Career Growth, for Diversity, and for Leadership, amongst others. To learn more and to view all our available career opportunities, please visit us at our website.
Genesis10 is an Equal Opportunity Employer. Candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.
