Quality Assurance (AI)

Job Title: Quality Assurance (AI) – Automation
Location: Hartford, CT (Hybrid – 3 days onsite per week)
Employment Type: Contract
Hands-on experience with Cyara is mandatory and must be clearly highlighted throughout the resume

Position Summary

The Aetna IT Delivery organization within CVS Health’s Data, Digital, Analytics, and Technology (DDAT) division is seeking a Quality Assurance Automation Engineer (AI) to design, develop, and maintain automated testing solutions for virtual assistants and conversational AI platforms. These solutions support members and providers by delivering accurate and timely health insurance information across multiple digital channels.

The ideal candidate will bring strong expertise in QA automation, BDD/TDD methodologies, conversational AI testing, and healthcare systems.

Required Qualifications

• 3+ years of experience in QA automation and software testing

• Strong programming fundamentals (OOP, data structures) with hands-on experience in Java, JavaScript, or Python

• Proven experience designing and maintaining scalable automation frameworks

• Mandatory hands-on experience with:

  • Cyara

  • Hammer On Demand

  • Selenium

• Solid understanding of SDLC, Agile methodologies, and QA best practices

• Experience with CI/CD tools and version control systems (Git)

• Expertise in:

  • BDD (Cucumber, SpecFlow)

  • TDD

  • Functional, Regression, Integration, and Performance Testing

  • Backend API test automation and database testing

• Healthcare domain knowledge, including HIPAA compliance

• Exposure to GenAI and AI-driven testing tools and their integration into QA processes

• Strong analytical, problem-solving, and communication skills

Preferred Qualifications

• Hands-on experience testing conversational AI, chatbots, and voice-based virtual assistants

• Familiarity with AI model evaluation metrics (accuracy, precision, recall, F1-score)

• Experience with AI-powered IDEs / Agentic tools (e.g., Windsurf, Cursor, or similar)

• API testing experience with REST and SOAP

• Knowledge of cloud platforms such as AWS, Google Cloud Platform, or IBM Cloud