Senior QA Engineer - Conversational AI Testing

Jacksonville, FL, US • Posted 6 hours ago • Updated 6 hours ago
Contract W2
On-site
USD $47.86 - 52.86 per hour
Company Branding Image
Fitment

Dice Job Match Score™

🛠️ Calibrating flux capacitors...

Job Details

Skills

  • JAX
  • Test Plans
  • Workflow
  • Regression Testing
  • Access Control
  • Documentation
  • Collaboration
  • Product Engineering
  • Machine Learning (ML)
  • Language Models
  • Selenium
  • Cypress
  • POSTMAN
  • Python
  • Java
  • JavaScript
  • API QA
  • Continuous Integration
  • Continuous Delivery
  • Natural Language Processing
  • Analytical Skill
  • Conflict Resolution
  • Problem Solving
  • Communication
  • Vertex
  • Microsoft Azure
  • Artificial Intelligence
  • HIPAA
  • System On A Chip
  • Quality Assurance
  • Agile
  • Scrum
  • Health Care
  • Financial Services
  • Test Cases
  • Automated Testing
  • Scripting
  • API
  • Integration Testing
  • Load Testing
  • Defect Tracking
  • Reporting
  • SQL
  • Database
  • Version Control
  • Git
  • Prompt Engineering
  • Regulatory Compliance
  • Testing
  • MEAN Stack
  • Customer Service
  • Training And Development
  • SAP BASIS

Summary

Software Guidance & Assistance, Inc., (SGA), is searching for a Senior QA Engineer - Conversational AI Testing for a contract assignment with one of our premier Healthcare Services clients in Jacksonville, FL . (Occasional onsite visits to Jax Office)

Responsibilities :
  • Lead quality assurance efforts for our conversational AI applications. This role requires deep expertise in testing AI-driven systems, prompt engineering validation, and test automation. You will be responsible for designing comprehensive test strategies, identifying edge cases in AI behavior, and ensuring our conversational AI products meet the highest standards of reliability, accuracy, and security.
  • AI Application Testing: Design and execute test plans specifically for conversational AI systems, including chatbots, virtual assistants, and language model-based applications
  • Prompt Engineering Validation: Collaborate with AI/ML teams to test and validate prompt effectiveness, consistency, and safety across various user scenarios
  • Test Automation Development: Build and maintain automated test suites for conversational AI workflows, including API testing, response validation, and regression testing
  • Quality Strategy: Develop QA strategies that address unique challenges of AI testing, including non-deterministic outputs, hallucination detection, and bias identification
  • Security & Compliance: Ensure testing aligns with organizational security standards, including HIPAA compliance (Standard 4500), information protection requirements (Standard 5110), and access control protocols
  • Documentation & Reporting: Create detailed test cases, defect reports, and quality metrics; communicate findings to stakeholders
  • Cross-functional Collaboration: Work with product, engineering, and AI/ML teams to understand requirements and validate solutions
Required Skills :
  • Bachelor's Degree in an IT related field or relevant work experience
  • 5+ years of QA experience, with at least 2+ years testing AI/ML-based applications or conversational systems
  • 5-8 years of Professional experience in an IT related field preferably in a development or testing environment
  • Prompt Engineering Expertise: Demonstrated experience designing, testing, and validating prompts for language models; understanding of prompt injection, jailbreaking, and safety considerations
  • Test Automation Proficiency: Strong hands-on experience with test automation frameworks and tools (e.g., Selenium, Cypress, Postman, REST Assured, or equivalent); ability to write clean, maintainable automation code
  • Technical Skills: Proficiency in at least one programming language (Python, Java, JavaScript, or similar); familiarity with API testing and CI/CD pipelines
  • AI/Conversational AI Knowledge: Understanding of how conversational AI systems work, including LLMs, NLP concepts, and common failure modes
  • Analytical Mindset: Strong problem-solving skills with ability to identify edge cases and non-obvious failure scenarios in AI systems
  • Communication: Excellent written and verbal communication skills; ability to explain technical QA concepts to non-technical stakeholders
Preferred Skills :
  • Experience testing multiple AI platforms (OpenAI, Google Vertex AI, Azure OpenAI, Anthropic Claude, etc.)
  • Familiarity with AI safety testing frameworks and bias detection methodologies
  • Experience with performance and load testing for AI applications
  • Knowledge of HIPAA, SOC 2, or other healthcare/regulated industry compliance requirements
  • Certification in QA, test automation, or related field
  • Experience with agile/scrum methodologies
  • Background in healthcare, financial services, or other regulated industries
  • Technical Competencies
  • Test case design and execution
  • Test automation and scripting
  • API and integration testing
  • Performance and load testing
  • Defect tracking and reporting
  • SQL and database query basics
  • Version control systems (Git)
  • Prompt engineering and LLM interaction
  • Security and compliance testing
SGA is a technology and resource solutions provider driven to stand out. We are a women-owned business. Our mission: to solve big IT problems with a more personal, boutique approach. Each year, we match consultants like you to more than 1,000 engagements. When we say let's work better together, we mean it. You'll join a diverse team built on these core values: customer service, employee development, and quality and integrity in everything we do. Be yourself, love what you do and find your passion at work. Please find us at .

SGA is an Equal Opportunity Employer and does not discriminate on the basis of Race, Color, Sex, Sexual Orientation, Gender Identity, Religion, National Origin, Disability, Veteran Status, Age, Marital Status, Pregnancy, Genetic Information, or Other Legally Protected Status. We are committed to providing access, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, and our services, programs, and activities. Please visit our company to request an accommodation or assistance regarding our policy
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: sgainc
  • Position Id: 26-01943
  • Posted 6 hours ago

Company Info

About Software Guidance & Assistance

Founded in 1981, SGA is a technology and resource solutions provider with a national footprint and headquartered in the shadow of Wall Street. We’re a certified women-owned business. We provide contingent staffing, direct placement, and professional and managed services to transform businesses and evolve careers. We’re small enough to tailor our services to each client and big enough to deliver for some of the world’s largest employers. Our professionals are experts in areas such as IT, finance, accounting, risk, and clinical.

SGA provides contingent staffing, direct placement, and professional and managed services nationwide for Fortune 500 companies, mid-size businesses and select startups.

Our core skillsets include all areas of technology – business & data analysis, cyber & network security, database administration, development & architecture, infrastructure, program & project management, quality assurance & testing. We also deliver talent across professional business functions such as finance, accounting, risk, and clinical.

Our Professional & Managed Services team delivers IT projects through onshore, offshore and hybrid delivery models. We develop software products, modernize applications, add features, and integrate and maintain systems. Our scope covers, among others, complex application suites, data management and visualizations, machine learning and mobile applications.

About_Company_OneAbout_Company_Two
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Jacksonville, Florida

Today

Contract

USD 47.00 - 52.00 per hour

Jacksonville, Florida

Today

Contract

USD 60.00 - 65.00 per hour

Jacksonville, Florida

Today

Contract

USD 88.00 - 88.00 per hour

Jacksonville, Florida

Today

Contract

USD 60.00 - 65.00 per hour

Search all similar jobs