Automation AI Tester

Role: Automation AI Tester

Location: Plano, TX

Duration: 12+ Months

Job Description:

Looking for an experienced Automation, Performance & AI Testing Lead to define and drive quality engineering across our products and services. You’ll lead test automation and performance engineering while helping teams apply AI and build agentic workflows that improve software delivery and quality. This role combines hands-on technical leadership with coaching, planning, and collaboration across engineering, product, and delivery teams.

Key responsibilities

• Define and evolve Avertack’s automation, performance, and AI-enabled quality engineering strategy.
• Design and develop maintainable automated test frameworks for UI, API, integration, and end-to-end testing.
• Establish performance engineering practices, including load, stress, endurance, and scalability testing.
• Identify performance bottlenecks and work with engineering teams to diagnose and resolve them.
• Design, build, and implement agentic workflows that automate testing, triage, test-data preparation, defect analysis, or other quality engineering processes.
• Evaluate AI agents, large language models, and orchestration approaches; apply them responsibly with appropriate validation, human oversight, and security controls.
• Integrate automated tests and AI-enabled workflows into CI/CD pipelines and advise on quality gates and release readiness.
• Define quality metrics and reporting for test coverage, reliability, defects, performance, and AI workflow effectiveness.
• Guide test planning, risk assessment, test data management, and environment strategy.
• Collaborate with architects, developers, product managers, and operations teams throughout the software lifecycle.
• Mentor engineers and promote shared ownership of quality.

Required qualifications

• 10+ years of experience in software testing, quality engineering, and test automation, including substantial technical leadership.
• Proven experience building and scaling automation frameworks for APIs and web applications.
• Strong programming or scripting skills in one or more languages such as Java, Python, JavaScript/TypeScript, or C#.
• Practical experience with tools such as Playwright, Selenium, Cypress, REST Assured, Postman, JMeter, Gatling, or equivalent.
• Experience designing performance tests, interpreting results, and driving remediation.
• Hands-on knowledge of AI/LLM applications and the ability to build agentic workflows, including tool use, workflow orchestration, and integrating agents with APIs or enterprise systems.
• Ability to evaluate AI-generated results and account for reliability, privacy, security, and human review in AI-enabled workflows.
• Experience with CI/CD systems and integrating automated testing into delivery pipelines.
• Strong understanding of test strategy, software development lifecycle, defect management, and quality metrics.
• Experience leading teams, mentoring engineers, and influencing quality practices across teams.

Preferred qualifications

• Experience with agent frameworks, AI orchestration platforms, retrieval-augmented generation (RAG), or evaluation frameworks.
• Experience with cloud-native, distributed, microservices, or event-driven systems.
• Familiarity with observability tools, application monitoring, containers, Kubernetes, and infrastructure-as-code.
• Knowledge of security and accessibility testing principles.
• Experience in Agile or DevOps delivery environments