QA Engineer-Artificial Intelligence

  • McLean, VA
  • Posted 1 day ago | Updated 2 hours ago

Overview

On Site
$55 - $70 per hour
Contract - W2
Contract - 6+ Month(s)

Skills

QA Engineer-Artificial Intelligence

Job Details



We have partnered with our client in their search for a


Responsibilities


Job Summary:



As an QA Engineer within our Client's healthcare AI & Analytics team, you will ensure the accuracy, reliability, and validity of our AI, ML, and Generative AI (GenAI) products and services. Leveraging your expertise in quality assurance methodologies, you will perform Functional, UI, and API testing to effectively validate products such as AI chatbots, ML/GenAI-generated clinical documents, microservices, and other software solutions. Your role will tackle unique challenges in GenAI environments, including output quality assurance and hallucination detection, ensuring stringent standards are consistently met. Aligned with this, you will support and contribute to our maturing AI Inter-Rater Reliability (AI-IRR) process and services.



Collaborating closely with data scientists, software developers, engineers, and stakeholders, you will design and execute comprehensive test plans and strategies specific to our AI and GenAI platforms. Responsibilities include creating and refining detailed test cases, verifying prompt-response accuracy, and ensuring the ethical and responsible use of AI outputs, ultimately contributing to improved patient outcomes.



Responsibilities:



  • Test Planning & Execution: Develop and implement test strategies, detailed test plans, and test cases tailored to AI, ML, and GenAI solutions

  • Quality Assurance & UAT: Conduct thorough Functional, UI, and API testing, including GenAI output quality, hallucination detection, and user acceptance testing to ensure optimal application performance

  • Collaboration: Work closely with data scientists, software developers, engineers, and project managers to validate algorithms and integrate QA best practices into development processes

  • Automation & Optimization: Implement automation testing frameworks to optimize testing efforts, including automated GenAI evaluation workflows

  • Data Integrity & Bias Detection: Verify training data integrity, perform bias detection, and conduct prompt-response validations for AI and GenAI model training

  • AI Inter-Rater Reliability: Review and verify AI-generated outputs for consistency and accuracy. Collaborate with AI/ML engineers to enhance automated evaluation systems and pipelines

  • Documentation & Reporting: Document test results, identify defects, collaborate to resolve issues, and prepare comprehensive reports with actionable insights for continuous improvement



Education:



  • Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field; equivalent work experience will be considered


Experience:



  • 3+ years of core quality assurance experience (preferably at the enterprise level, managing large scale projects), with a strong background in traditional QA and UAT methodologies

  • Exposure to AI/ML: some experience in AI or Machine Learning (ML)

  • Proficiency in RESTful APIs and testing tools such as Postman, specifically for testing GenAI model APIs like OpenAI and Hugging Face

  • Experience verifying training data integrity, bias detection, and prompt-response validation for AI models

  • Familiarity with data annotation tools, and quality control processes

  • Demonstrated experience in testing complex software applications

  • Proven experience designing, documenting, and executing comprehensive test plans for AI-driven products

  • Strong analytical thinking, attention to detail, excellent communication skills, and the ability to thrive in a fast-paced, cross-functional team environment


Technical Skills:



  • Strong experience with QA tools and frameworks like PyTest, Selenium, or JUnit

  • Solid knowledge of software development life cycle (SDLC) and agile methodologies

  • Understanding of AI and ML technologies

  • Solid knowledge of Python, with experience or knowledge in Gen AI libraries like LangChain and LlamaIndex (preferred)

  • Familiarity with data annotation tools and quality control processes for GenAI-specific tasks

  • Knowledge in testing AI, ML, and Generative AI outputs, including Functional, UI, and API testing

  • Knowledge of GenAI-specific annotation for RAG and LLM fine-tuning


Preferred Qualifications



  • Experience with GenAI model evaluation metrics such as perplexity and BLEU

  • Expereince with Gen AI evaluation frameworks like RAGAS, and GenAI concepts, algorithms, and output validation techniques

  • Experience in healthcare technology or data analytics environments

  • Familiarity with regulatory and ethical standards related to AI and healthcare solutions


Title QA Engineer


Location McLean, VA


Client Industry Healthcare


Compensation $60-70/hr


Ref ID 1681715




About Korn Ferry


Korn Ferry unleashes potential in people, teams, and organizations. We work with our clients to design optimal organization structures, roles, and responsibilities. We help them hire the right people and advise them on how to reward and motivate their workforce while developing professionals as they navigate and advance their careers. To learn more, please visit Korn Ferry at ;/span>

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

About Korn Ferry