Overview
Remote
Depends on Experience
Full Time
Skills
API
Accessibility
Amazon Web Services
Analytical Skill
Apache JMeter
Artificial Intelligence
Authentication
Authorization
Automated Testing
Clarity
Cloud Computing
Continuous Delivery
Continuous Integration
Data Extraction
Data Security
Debugging
Documentation
Generative Artificial Intelligence (AI)
Good Clinical Practice
Google Cloud
Google Cloud Platform
Machine Learning (ML)
OAuth
POSTMAN
Performance Testing
Privacy
Product QA
Prompt Engineering
Quality Assurance
RBAC
Regulatory Compliance
SaaS
Switches
Testing
UI
User Experience
WCAG
Workflow
Job Details
We are seeking a meticulous QA Engineer with hands-on experience in testing AI-driven platforms to join our dynamic team. This role focuses on validating generative AI models (like Gemini, PaLM), API integrations, data security, and user experience across web-based AI studio environments.
You ll play a critical role in ensuring functional reliability, ethical compliance, and top-tier performance across AI workflows including prompt/response behaviors, API validation, and UI responsiveness.
Key Responsibilities
1. Functional Testing
Skills & Requirements
Bonus Points
You ll play a critical role in ensuring functional reliability, ethical compliance, and top-tier performance across AI workflows including prompt/response behaviors, API validation, and UI responsiveness.
Key Responsibilities
1. Functional Testing
- Validate core AI functionalities including model loading, prompt input/output, and token calculation.
- Test model switching, fine-tuning, and prompt template saving features.
- Verify integrations with tools like Google Sheets, Docs, and BigQuery.
- Test AI responses for clarity, creativity, context retention, summarization, and data extraction.
- Validate ethical response behavior and proper citation when applicable.
- Verify that no sensitive user data is retained or leaked.
- Validate RBAC controls, prompt history access, and secure integration with Google Cloud Identity/IAM.
- Ensure REST/gRPC APIs return accurate responses, handle malformed requests, and follow authentication/authorization flows.
- Test rate limiting, throttling, and latency SLAs.
- Validate system responses to invalid or oversized inputs, model unavailability, and try again workflows.
- Test user feedback mechanisms like thumbs up/down.
- Conduct load tests (100+ sessions), stress tests, and performance validation across browsers/devices.
- Ensure prompt response within <2 seconds under normal load.
- Validate prompt rendering, syntax highlighting, mobile responsiveness, dark/light mode, and WCAG compliance.
- Test model safeguards against offensive, unsafe, or unethical prompts.
- Ensure AI-generated content includes appropriate disclaimers and bias mitigation.
Skills & Requirements
- 5+ years in QA Testing, preferably in AI/ML product environments.
- Experience with API tools (Postman, Swagger, REST/gRPC testing).
- Familiarity with LLM-based platforms like Gemini, Bard, PaLM, or GPT.
- Experience with performance testing tools (e.g., JMeter, Locust).
- Knowledge of security protocols (RBAC, OAuth, IAM) and compliance standards.
- Strong understanding of ethical AI principles.
- Excellent analytical, documentation, and debugging skills.
Bonus Points
- Experience working with AI/ML prompt engineering or fine-tuning.
- Prior QA testing in SaaS or Cloud platforms (Google Cloud Platform/AWS preferred).
- Exposure to CI/CD pipelines and automated testing frameworks.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.