Chatbots Automation Tester

Hybrid in East Hartford, CT, US • Posted 2 days ago • Updated 2 days ago

Contract Independent

Contract W2

No Travel Required

Hybrid

Depends on Experience

Fitment

Dice Job Match Score™

📋 Comparing job requirements...

Job Details

Skills

Hammer On Demand
Cyara
Selenium

Summary

Job Description:

Conversational AI Testing: Hands-on experience or strong exposure to testing chatbots and voice-based virtual assistants across multiple platforms.
Hands-on experience with automation tools including Hammer On Demand, Cyara and Selenium
Healthcare Domain Knowledge: Familiarity with HIPAA compliance and healthcare workflows
AI Model Evaluation: Familiarity with key evaluation metrics such as accuracy, precision, recall, and F1-score for assessing AI model performance.

Behavior Driven Development and Test Driven Development methodologies, automation scripting, and multi-channel solution (voice & chat) knowledge.

Position Summary
The Aetna IT Delivery organization, part of CVS Health’s Data, Digital, Analytics, Technology (DDAT) division, is seeking a highly skilled Quality Assurance Automation Engineer to design, develop, and maintain automated testing solutions for virtual assistants (agents) to support members and providers by accessing timely, accurate health insurance answers across digital channels.
The ideal candidate will have strong experience in Behavior Driven Development and Test Driven Development methodologies, automation scripting, and multi-channel solution (voice & chat) knowledge.

Key Responsibilities
• Test Automation Development:
• Design, develop, and maintain automated test scripts using industry-standard tools and frameworks (e.g., Selenium, Cypress, Playwright, Appium).
• Build and enhance automation frameworks to support functional, regression, and performance testing.
• Quality Assurance:
• Collaborate with developers and QA analysts to understand application functionality and define test coverage.
• Execute automated test suites and analyze results to identify defects and ensure compliance with quality standards.
• Continuous Integration & Delivery (CI/CD):
• Integrate automated tests into CI/CD pipelines using tools like Jenkins, GitLab CI, or Azure DevOps.
• Monitor and maintain test execution in CI environments to ensure timely feedback.
• Defect Management:
• Log, track, and verify defects using tools such as JIRA or Azure DevOps.
• Work with development teams to reproduce issues and validate fixes.
• Documentation & Reporting:
• Prepare detailed test plans, test cases, and automation scripts documentation.
• Generate and present test execution reports to stakeholders.
• Process Improvement:
• Identify opportunities to improve automation coverage and efficiency.
• Stay updated with emerging trends, tools, and best practices in test automation.

Required Qualifications
• 3+ years of experience in QA automation and software testing.
Strong technical expertise in core programming concepts (e.g., Java OOPs, Data Structures) with the ability to adopt and work with multiple languages such as Java, JavaScript, or Python.
• Hands-on experience implementing and enhancing test automation frameworks, including designing scalable and maintainable solutions.
• Conversational AI Testing: Hands-on experience or strong exposure to testing chatbots and voice-based virtual assistants across multiple platforms.
Hands-on experience with automation tools including Hammer On Demand, Cyara and Selenium
Strong understanding of SDLC, Agile methodologies, and QA principles.
• Experience with CI/CD tools and version control systems (e.g., Git).
• Frameworks & Methodologies:
• BDD (e.g., Cucumber, SpecFlow)
• TDD
• Testing Expertise: Functional, Regression, Integration, and Performance Testing, including proficiency in backend API test automation and database testing

Exposure to GenAI and AI-driven tools utilized for test automation, with an understanding of how these can be integrated into QA processes
Strong analytical and problem-solving skills
Excellent communication and collaboration abilities.
• Ability to clearly articulate technical concepts and demonstrate practical problem-solving skills during discussions.
• Healthcare Domain Knowledge: Familiarity with HIPAA compliance and healthcare workflows
• AI Model Evaluation: Familiarity with key evaluation metrics such as accuracy, precision, recall, and F1-score for assessing AI model performance.
Experience with Agentic IDEs: Familiarity with AI-powered development environments (e.g., Windsurf, Cursor, or similar) for creating, maintaining, and optimizing automated test scripts.
Experience with API testing (REST, SOAP)
• Knowledge of cloud platforms (AWS, Google Cloud Platform, or IBM Cloud)
Education
Bachelor’s degree in Computer Science, Engineering, or related field.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91172358
Position Id: 8887624
Posted 2 days ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Chatbots Automation Tester

Dice Job Match Score™

Job Details

Skills

Summary

Similar Jobs