AI Auotomation Test Engineer

Title: AI Test Engineer – Automation with Manufacturing  domain 

Location: Richmond, VA — on-site required, with regular access to the manufacturing floor, may test UST Nashville code bases as well

Position Summary
Client is seeking two Automated Test Engineers to build fully automated, end-to-end test coverage for manufacturing applications. The centerpiece of this role is complete process automation: Playwright-driven UI and workflow testing combined with simulation of PLC messages from the control system, so that entire shop-floor scenarios — from control-system event to application response — can be exercised automatically without touching production equipment.
This role also covers AI/LLM quality: designing evaluation frameworks for AI behavior, building automated test suites for agentic applications, and participating directly in the development cycle — including re-prompting and tuning AI behavior when tests reveal gaps.
These roles are on-site in Richmond because you cannot simulate a control system You will walk the floor, observe the real processes and equipment
whose messages you are simulating, and work directly with operators and controls engineers to make sure test scenarios match reality.
These are flexible, well-rounded positions not tied to any single project or department. 
While manufacturing is the primary focus, assignments may extend to HR initiatives or other enterprise AI efforts.
Key Responsibilities
End-to-End Test Automation (Primary Focus)
· Build and maintain automated test suites covering full application workflows — UI, API, and database validation — using the best-fit tool for each application
(Playwright is the current standard, but the role requires fluency across the testing landscape, not loyalty to one tool)
· Design and build PLC message simulation so control-system inputs can be generated on demand: simulate tag changes, equipment events, alarms, and message sequences as they would arrive from the plant floor (e.g., via OPC UA, MQTT, TCP/IP sockets, file drops, or middleware queues, depending on the interface)

· Compose the two into fully automated end-to-end scenarios: simulated PLC event → application/MES behavior → verified outcome — no manual steps, runnable on every build
· Build test scenario libraries with operators and controls engineers so simulated message sequences reflect real production behavior, including failure modes (out-of- order messages, dropped connections, bad data)
AI & Application Testing
· Design and build automated test suites for AI-powered applications: unit, integration, regression, and end-to-end tests
Development Cycle Participation 
· When tests reveal AI behavior gaps, participate in fixing them: refine prompts, adjust context strategies, and re-test (re-prompting is part of this job, not someone else's) 
· Build and maintain CI/CD test integration so quality gates run automatically on every change 
· Document test strategies, evaluation criteria, and known limitations for each application Enterprise Enablement
· Work with Infrastructure, Security, and Identity teams to complete these requests
· Follow Client change management and SDLC governance processes

Production Support
· Support applications after go-live: reproduce reported issues, verify fixes, and update or create test cases in response to issues that arise in production — every
incident should leave the regression suite stronger than before · Maintain and evolve test suites as applications change, so coverage stays
current with the application rather than decaying after go-live · Participate in ServiceNow-based incident and change workflows

Required Qualifications
· Bachelor's degree in Computer Science, Information Systems, or equivalent practical experience

· 3+ years of professional software development experience (application code, not only test code) in addition to test automation experience — this role builds
simulators, harnesses, and application features, not just test scripts · Broad familiarity with the test automation landscape — knowledge and
proficiency with Playwright plus working knowledge of other major platforms (e.g., Selenium, Cypress, pytest, xUnit/NUnit, Postman/API test tools) and the judgment to pick the right tool for each application; proficiency in at least one programming language (Python, C#, or JavaScript/TypeScript)
· Experience with industrial control system integration or simulation — PLC communications (OPC UA, Modbus, EtherNet/IP), message simulation/stubbing, or equivalent event-driven integration testing where the test harness generates the upstream system's messages
· Experience testing APIs and data-driven applications, including SQL-based validation
· Willingness and ability to work on-site in Richmond, VA and go to the manufacturing floor regularly — validating AI outputs against the real process, with the operators who run it, is a core part of this role
· Experience with integrating multiple systems (i.e. Blue Yonder, Middleware, SAP) Preferred

Qualifications
· Direct experience with MES testing or PLC/SCADA environments (Rockwell/Allen-Bradley, Siemens, Kepware, or similar) — strongly preferred given the manufacturing focus of these roles
· Experience building simulators, emulators, or hardware-in-the-loop test rigs for control systems
· Familiarity with enterprise IT processes: firewall requests, AD administration, test environment provisioning, change management (ServiceNow)