Software Development Engineer in Test With AI Experience (10+ Years) (NEED LOCAL TO NJ ONLY) (W2 ROLE)

Title: Software Development Engineer in Test With AI Experience (10+ Years)

Location: Jersey City, NJ

Type: Contract

Interview:: TEAMS (1^ST ROUND) / IN PERSON (2^nd Round)

Job Description: Role Summary

We are seeking a self-driven Software Development Engineer in Test (SDET II) with experience testing AI-powered applications, Generative AI solutions, and Agentic AI systems. This role is responsible for designing and executing quality engineering activities, developing automation solutions, validating APIs and integrations, supporting CI/CD and release readiness, and promoting a quality-first engineering mindset across traditional and AI-driven platforms.

Key Experience Requirements

•       8+ years of software quality engineering, test automation, or SDET experience in Agile development environments.

•       Strong proficiency in Python, Pytest, Pandas, and object-oriented programming principles.

•       6+ years of experience with UI automation frameworks such as Playwright or Selenium.

•       6+ years of experience testing RESTful APIs and microservices.

•       Hands-on experience with CI/CD pipelines and DevOps practices.

•       Experience working with SQL and NoSQL databases.

•       Experience with AWS or other cloud-based platforms.

•       Understanding prompt engineering, prompt validation, hallucination detection, bias testing, guardrail validation, and AI output evaluation.

•       Knowledge of LLMs, Retrieval-Augmented Generation (RAG), AI agents, MCP, and AI-assisted workflows.

•       Ability to develop QA strategies, test frameworks, and automation solutions for AI-driven applications.

•       Analytical, problem-solving mindset with continuous learning and quality-first engineering habits.

Key Responsibilities

•       Design and execute functional, integration, system, regression, and automation testing activities.

•       Apply Shift-Left testing practices by participating in requirement reviews and defining test scenarios early in the SDLC.

•       Build, maintain, and enhance UI automation frameworks using Python, PyTest, and Playwright.

•       Perform manual and automated API testing using industry-standard tools and frameworks.

•       Collaborate with developers, product owners, business analysts, and DevOps teams to ensure product quality.

•       Work with SQL and NoSQL databases for backend validation and data testing.

•       Support CI/CD pipelines and ensure automated tests are integrated into deployment processes.

•       Create test plans, test cases, test data strategies, and quality metrics dashboards.

•       Manage defect lifecycle activities and provide clear reporting of quality risks.

•       Develop AI test datasets, benchmarks, evaluation frameworks, and automated validation approaches.

•       Experience with performance and load testing using JMeter or similar tools.

•       Knowledge of AI observability, model monitoring, and evaluation platforms.

•       Experience with monitoring tools such as New Relic, Datadog, Splunk, or CloudWatch.

Preferred Qualifications

•       Experience testing Generative AI and Agentic AI applications.

•       Ability to validate AI outputs for accuracy, reliability, and business relevance.

•       Knowledge of AI agent workflows, prompt testing, hallucination detection, and model evaluation.

•       Understanding AI safety, guardrails, security, compliance, and responsible AI practices.

•       Experience with LangChain, LangGraph, MCP, OpenAI, Anthropic, Azure OpenAI, Kiro or Amazon Bedrock.

•       Familiarity with AI governance, risk management, and AI quality metrics.