AI Quality Engineer

Role: AI Quality Engineer

Location: Saint Louis, MO 

Mandatory Areas: Drift Detection, UIPath, Google kubernetes Engineering

What You'll Do

The responsibilities of an AI Quality Engineer are split between testing the AI itself, AI-Driven Automation and using effective Prompt engineering to improve testing and Quality processes. This role also involves engaging and collaborating with cross-functional teams.

Your key responsibilities will include:

• Define and execute end-to-end testing and validation frameworks for AI services, encompassing machine learning models, agentic workflows, and automation solutions. Focus on rigorous model and data validation, real-time observability, and the implementation of autonomous test generation.
• Data Quality Auditing: Validating training and testing datasets for bias, noise, and completeness.
• Model Performance Testing: Defining and tracking metrics beyond simple &; pass/fail & ; such as Precision-Recall curves, F1-score, and Mean Absolute Error (MAE).
• Adversarial Testing: Intentionally providing & ;prompt injections or perturbed images to see if the model breaks or leaks sensitive data.
• Drift Detection: Monitoring production models for concept drift (changes in real-world data patterns) and data drift. Bias
• Fairness Monitoring: Ensuring the model's outputs remain ethical and do not discriminate against specific demographics over time.
• You will be the primary gatekeeper for AI Governance and compliance, ensuring the model adheres to Equifax AI standards.
• Design comprehensive test cases and strategic test plans to validate RPA solutions (UiPath) for both new implementations and existing systems.
• Document detected defects and manage their lifecycle through to resolution.
• Validate the Hand-off points where an AI Agent triggers an PRA bot to perform a structural task.

Recommended Qualification and Experience:

• 5+ years in Software Quality Engineering, with at least 2 years focused on AI/ML, UiPath automation testing.
• Understanding of OCR (Optical Character Recognition), NLP (Natural Language Processing), and LLM architectures.

Value Add Experience

• Possesses prior experience in and Python scripting, as well as experience with Salesforce and Google Cloud Platform (Google Cloud Platform).
• Proficient in system integrations utilizing technologies such as SQL, REST APIs, and Google Kubernetes Engine (GKE).