Apply Now

AI Quality Analyst-# 26-22273

Hybrid in Lincolnshire, IL, US • Posted 1 hour ago • Updated 1 hour ago

Contract W2

5 Months

Hybrid

$55 - $56/hr

Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

QA ENGINEER
QUALITY ASSURANCE ENGINEER
QA ANALYST
TEST ENGINEER
SOFTWARE TEST ENGINEER
SDET
TEST AUTOMATION ENGINEER
QUALITY ENGINEER
DATA QUALITY ANALYST
DATA ANALYST
AI QA ENGINEER
AI TEST ENGINEER
ML TEST ENGINEER
AI/ML
ARTIFICIAL INTELLIGENCE
MACHINE LEARNING
DATA
PYTHON
SCRIPT
TEST AUTOMATION
AUTOMATION
SELENIUM
PLAYWRIGHT
CYPRESS
REST-ASSURED

Summary

Duration: 5+ Months Contract
Hybrid Position

Job Description:

The AI Quality Analyst is a critical role responsible for ensuring the performance, safety, and reliability of our cutting-edge AI/ML models. You will be at the forefront of our development lifecycle, designing and executing comprehensive evaluation strategies to identify model weaknesses, potential biases, and critical edge cases. This role requires a blend of analytical rigor, technical aptitude, and a deep curiosity for how AI models behave in real-world scenarios. You will not just find bugs, but provide the actionable insights that drive model improvement and guide our research and development efforts.

Responsibilities:

Evaluation Strategy & Benchmark Development: Design, develop, and maintain a comprehensive suite of test cases and evaluation benchmarks. Proactively identify potential model failure points, including edge cases, adversarial inputs, and sources of bias.
Error Analysis & Failure Triage: Conduct systematic error analysis to categorize model failures and identify underlying patterns. Triage defects, prioritize them based on severity and impact, and work with the development team to ensure resolution.
Data Sourcing & Curation: Source, curate, and manage high-quality datasets for model evaluation and testing. This includes performing data annotation and validation to ensure the integrity of our ground-truth data.
Exploratory & Adversarial Testing (Red Teaming): Perform unscripted, exploratory testing to Client unexpected model behaviors. Participate in red teaming exercises to intentionally challenge our models and identify potential safety and security vulnerabilities.
Test Environment Management: Set up, maintain, and troubleshoot testing and demonstration environments to ensure a stable and reliable evaluation pipeline.
Reporting & Insights: Analyze and synthesize test results into clear, actionable reports for both technical and non-technical stakeholders. Translate complex findings into concrete recommendations for model improvement.
Process Improvement: Actively participate in post-hoc evaluation reviews and contribute to the continuous improvement of our testing methodologies, tools, and overall quality assurance processes.

Experience:

Proven experience in a quality assurance, testing, or data analysis role, preferably within the AI/ML domain.
A deep understanding of the machine learning lifecycle and the common failure modes of AI models.
Hands-on experience with data annotation, data validation, and managing large datasets.
Meticulous attention to detail and a methodical approach to problem-solving.

Skillsets:

Strong analytical skills with the ability to identify patterns in data and draw meaningful conclusions.
Expertise with industry-standard test automation tools and libraries (e.g., Selenium, Playwright, Cypress, REST-assured).
Experience in testing across different platforms (e.g. comprehensive testing of mobile Android/iOS and web applications)
Experience with bug tracking systems (e.g., Jira) and test case management tools.
Scripting skills (e.g., Python) for test automation and data manipulation.
(Preferred) experience in testing AI systems, including evaluating agentic responses, model performance metrics, and data integrity.
Excellent communication skills, with the ability to clearly document bugs and articulate complex technical issues.
Familiarity with computer vision or other specific AI domains relevant to our work.

Education:

Bachelors Degree

About US Tech Solutions:
US Tech Solutions is a global staff augmentation firm providing a wide range of talent on-demand and total workforce solutions. To know more about US Tech Solutions, please visit .

US Tech Solutions is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, colour, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.

AI Statement: By applying, you acknowledge that AI-assisted tools may be used during hiring.

#LI-AS140

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: usts
Position Id: 26-22273
Posted 1 hour ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

EIS QA Engineer II

Lincolnshire, Illinois

•

Today

ApTask Global Workforce (AGW) is seeking a AI Quality Analyst with an information technology company. This is a 5+ month contract opportunity with our client located in Lincolnshire, IL. This is a hybrid position. Summary: The AI Quality Analyst is a critical role responsible for ensuring the performance, safety, and reliability of our cutting-edge AI/ML models. You will be at the forefront of our development lifecycle, designing and executing comprehensive evaluation strategies to identify mo

Full-time

ETL Tester - Claims

Rosemont, Illinois

•

Yesterday

Job Description: Our client is seeking an ETL Tester - Claims. In this position, Sr. Data Quality Tester will be responsible for ETL process Data testing. The successful candidate will deeply understand the needs of the organization and determine how to best test modeled data, as well as maintain and improve the health of our existing ETL processes. The Sr. Data Quality engineer is responsible for planning and executing manual and automated testing activities on new and existing products to ensu

Third Party, Contract

65 - 75

Data Quality Engineer

Chicago, Illinois

•

Today

Location(s) Alpharetta, Georgia, Birmingham, Alabama, Chicago, Illinois, Downers Grove, Illinois, Jacksonville, Florida, Remote-CT, Remote-NJ, Remote-OH, Remote-PA, Remote-RI, Remote-VA Details Kemper is one of the nation's leading specialized insurers. Our success is a direct reflection of the talented and diverse people who make a positive difference in the lives of our customers every day. We believe a high-performing culture, valuable opportunities for personal development and professional

Full-time

USD 99,000.00 - 164,800.00 per year

Test Automation Engineer

Chicago, Illinois

•

Today

Interactive Brokers Group, Inc. (NASDAQ: IBKR), a member of the S&P 500, is a global financial services company headquartered in Greenwich, Connecticut, with offices in over 15 countries. Through its affiliates, Interactive Brokers provides automated trade execution and custody of securities, commodities, foreign exchange, and prediction markets on over 170 markets in numerous countries and currencies. For more than four decades, Interactive Brokers has focused on technology, automation, and in

Full-time

USD 140,000.00 - 160,000.00 per year

Search all similar jobs

More jobs at U.S. Tech Solutions Inc. in Lincolnshire, IL