QA Test Engineer

JOB DESCRIPTION | QA Test Engineer | Rose International

Job Title: QA Test Engineer

Location: Alpharetta, GA

Duration: 7 months

Must-Have Skills

·        API

·        Automation

·        CI/CD tools

·        GIT

·        Java

·        LLM (Large Language Model)

·        Python

·        QA

Education: Bachelor’s in Computer Science, Engineering, Data/Information Systems, or equivalent practical experience. 

TOP 5 SKILLS REQUIRED: 

·        3+ years in QA automation or SDET-type work (adjust by level); 1+ year exposure to AI/LLM or ML-driven features is a plus. 

·        Strong test automation in Python and/or Java/TypeScript. 

·        We are a platform team, testing APIs for high performance, automation will be primary focus. 

·        Strong communication and analytical skills. 

·        AI/ML experience

ADDITIONAL SKILLS REQUIRED: 

·        Hands-on with frameworks/tools such as: UI: Playwright / Cypress / Selenium and API: pytest + requests, Postman/Newman, REST Assured 

·        CI/CD integration: Git, GitHub Actions/Jenkins/GitLab CI, test reporting, gating. 

·        Test design: equivalence partitioning, boundary testing, risk-based testing, defect triage. AI-Specific Testing Competencies (Key) 

·        LLM/application behavior testing: validating correctness when outputs are probabilistic. 

·        Evaluation strategies: golden datasets, scoring rubrics, human-in-the-loop reviews. 

·        Non-determinism handling: statistical assertions, repeated runs, variance thresholds. 

·        Prompt and regression management: versioning prompts, detecting prompt drift, replay tests. 

·        RAG testing (if applicable): retrieval quality (recall/precision), grounding checks, citation validation, doc freshness. 

·        Safety & quality checks: hallucination detection, toxicity/PII leakage checks, policy compliance tests. 

Data & Observability 

·        Ability to create and maintain test datasets (structured + unstructured), including edge cases. 

·        Familiarity with telemetry for AI systems: - logging prompts/outputs safely, traceability, correlation IDs - tools like OpenTelemetry, ELK/Splunk, Datadog/Grafana (any equivalent) 

·        Understanding of data privacy constraints (masking/redaction) and secure test data practices. 

·        API / Microservices / Cloud 

·        Comfortable testing distributed systems: microservices, async workflows, queues/events. 

·        Basic cloud proficiency (AWS/Azure/Google Cloud Platform) and containerization (Docker, optional Kubernetes). Performance & Reliability Testing (AI-Aware) 

·        Load/performance testing for inference endpoints (latency, throughput, concurrency). 

·        Cost-aware testing (token usage, rate limits, fallbacks). 

·        Resilience tests: retries, circuit breakers, model timeouts, degraded-mode behavior. 

Nice-to-Have Domain Knowledge 

·        Familiarity with NLP concepts (embeddings, context windows, temperature/top-p). 

·        Experience with AI tooling: LangChain/LlamaIndex, evaluation tools, model gateways. 

·        Knowledge of regulatory/security needs relevant to the telecom domain. 

·        Soft Skills / Ways of Working 

·        Strong communication

·        able to explain AI quality issues clearly to product and engineering. 

·        Comfortable partnering with data science/ML engineers and backend teams. 

·        Ownership mindset: building reusable test harnesses, improving quality metrics, preventing regressions.