Job Title: Software Quality Assurance Analysts and Testers
Location: Yonkers, NY
Employer: 4Consulting, Inc
Position Type: Full Time
Worksite: Remote
Position Summary
4Consulting, Inc is seeking a qualified Software Quality Assurance Analysts and Testers to support technology consulting, application development, systems analysis, testing, data management, cybersecurity, infrastructure, or project delivery services for client projects. The selected candidate will perform duties consistent with the client’s project requirements and company delivery standards.
Essential Duties and Responsibilities
The candidate will be responsible for:
Develop and execute test plans, test cases, and evaluation scenarios to validate conversational AI, RAG, and Agentic AI solutions against defined business and technical requirements.
Evaluate AI-generated responses for accuracy, relevance, consistency, completeness, reliability, and alignment with expected outcomes.
Perform negative, edge-case, and adversarial testing to identify weaknesses, unexpected AI behavior, content retention issues, and potential failures in system guardrails.
Track testing results, quality metrics, and defects; communicate findings and provide clear recommendations regarding solution quality and release readiness.
Minimum Qualifications
The position requires:
Education:
Bachelor’s degree in Computer Science, Information Technology, Engineering, Business, or related field, or equivalent combination of education and experience.
Experience:
Experience in software quality assurance and testing, including test planning, test case development, test execution, defect identification, and quality evaluation.
Hands-on experience testing AI/GenAI solutions, including conversational AI, large language models, RAG, or Agentic AI, is preferred.
Experience with AI output evaluation, negative testing, edge-case testing, adversarial testing, and validation of AI safeguards is preferred.
Technical Skills:
AI/GenAI Testing, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Agentic AI, Test Planning, Test Case Development, AI Output Evaluation, Adversarial Testing, Jira, Defect Tracking
Business / Domain Skills:
Experience supporting State Government Health and Human Services programs, particularly Medicaid eligibility, enrollment, and member services within public-sector environments.
Experience supporting Medicaid Eligibility and Client Management (MECM) modernization initiatives involving member and staff interactions, eligibility operations, and enterprise knowledge access.
Experience validating business rules, application workflows, interfaces, data, batch processes, and end-to-end transactions through functional, integration, regression, UAT, and end-to-end testing.
Experience evaluating AI-driven solutions in regulated environments, ensuring generated information is accurate, consistent, complete, appropriate, and aligned with business requirements and operational expectations.
Communication:
Ability to communicate with technical and business stakeholders.
Documentation:
Ability to prepare project documentation, status updates, and deliverables.
Work Authorization:
Must be legally authorized to work in the United States. Sponsorship may be considered where permitted by company policy and project requirements.
Certification Required
Not Specified
Travel Required
None Specified
Supervisory Role
No
Preferred Qualifications
Experience testing AI/ML or GenAI solutions, including conversational AI, RAG, Agentic AI, and LLM-based applications.
Familiarity with AI evaluation methodologies, quality metrics, guardrail validation, and techniques for identifying unexpected or unreliable AI behavior.
Experience using test management and defect tracking tools such as Jira, with strong analytical and problem-solving skills.
Equal Employment Opportunity Statement
4Consulting, Inc is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, sex, national origin, age, disability, veteran status, citizenship status, immigration status were protected by law, or any other legally protected status. All candidates will be evaluated based on job-related qualifications, experience, skills, availability, and ability to perform the essential duties of the position.
