Search Jobs | Dice.com

100 results (3 new)

Group by company

Senior Technical Lead, Autonomous-Agent Verification, Long-Term Contract

Benchmark IT- Technology Talent

Remote • Today

Easy Apply

Contract

Depends on Experience

Technical Lead

Dexian DISYS

Brooklyn, Ohio • Today

Easy Apply

Contract

$65 - $70 per hour

ServiceNowTechnical Lead / Architect

SRS Consulting Inc

Remote • 7d ago

Easy Apply

Third Party, Contract

Depends on Experience

Agentic AI Lead/ Architect (Python)

Envision Technology Solutions

Alpharetta, Georgia • 4d ago

Easy Apply

Full-time, Third Party, Contract

Commercial AI & Agentic Lead

The Intersect Group

Hybrid in Smyrna, Georgia • Today

Easy Apply

Contract

Depends on Experience

Lead AI Agent Engineer

EPAM Systems

Remote • Today

Full-time

Agentic Automation Lead - Frontier AI

Eli Lilly and Company

South San Francisco, California • Today

Full-time

USD 193,500.00 - 338,800.00 per year

Lead Technical Analyst

HAYS

Florida • Today

Easy Apply

Full-time

USD 80,000.00 - 100,000.00 per year

Technical Lead

Alpha Omega Integration LLC

Arlington, Virginia • Today

Full-time

USD 150,000.00 - 175,000.00 per year

Lead - Standards & Assurance, Technology Labs

JLL

Remote or Chicago, Illinois • Today

Full-time

USD 175,000.00 - 257,150.00 per year

Senior Lead Architect, Agentic AI Solutions Architect

JPMorgan Chase & Co.

Jersey City, New Jersey • Today

Full-time

USD 175,750.00 - 260,000.00 per year

AI Security and Assurance Technical Lead

Agile Defense

Tampa, Florida • Today

Full-time

VP, Compliance, Surveillance and Supervision Technology Engineering Lead

TD Bank Group

New York, New York • 17d ago

Full-time

USD 105,000.00 - 220,000.00 per year

ServiceNow Technical Lead

Ryder System

Coral Gables, Florida • Today

Full-time

Compensation information provided in the description

Artificial Intelligence / Machine Learning Lead Verification and Validation Engineer

Penn State

State College, Pennsylvania • Today

Full-time

USD 140,004.00 - 265,704.00 per year

Senior/Staff Design Verification Engineer, HBM

Micron Technology Inc

Richardson, Texas • Today

Full-time

AI Lead

Navigant Consulting

McLean, Virginia • Today

Full-time

Senior AI Engineer - Contract to Hire

Infolob Global, Inc.

Harrisburg, Pennsylvania • Today

Easy Apply

Contract

Depends on Experience

Technology Lead Business Analyst

Citi

Sioux Falls, South Dakota • Today

Full-time

USD 97,360.00 - 146,040.00 per year

ASIC / FPGA Design Verification Engineer - (Lead or Senior)

Boeing Company

El Segundo, California • Today

Full-time

USD 146,200.00 - 197,800.00 per year

Generative AI Senior Delivery Lead

Citi

Tampa, Florida • 6d ago

Full-time

USD 141,440.00 - 212,160.00 per year

Generative AI Senior Delivery Lead

Citi

Tampa, Florida • Today

Full-time

USD 141,440.00 - 212,160.00 per year

Technical Lead / Architect – Agentic AI

EazyML

Remote • 14d ago

Easy Apply

Full-time

Depends on Experience

AI Senior Lead / Technologist, Irvine, CA - Onsite

Floga technologies

Irvine, California • Today

Easy Apply

Contract, Third Party

Senior Autonomy Engineer

Impulse Space

Boulder, Colorado • Today

Full-time

USD 150,000.00 - 200,000.00 per year

Sr. Technical Lead Architect

Request Technology, LLC

McLean, Virginia • 6d ago

Easy Apply

Full-time

210,000 - 225,000

Senior Integration and Testing Lead

Metova Federal

Orlando, Florida • Today

Full-time

Sr. Staff Engineer - Verification & Validation Lead

Qualcomm Technologies

Novi, Michigan • Today

Full-time

USD 152,800.00 - 229,200.00 per year

Lead Agentic AI Engineer - Vice President

Citi

Irving, Texas • Today

Full-time

USD 125,760.00 - 188,640.00 per year

AI Tech Lead

Motion Recruitment Partners, LLC

Chicago, Illinois • Today

Easy Apply

Full-time

$150000 - $190000

Benchmark IT- Technology Talent

Job Details

Skills

Software Engineering
Technical Lead
Technical Architect
Python
Automating Testing
ML-Operations
LLM's
Agent Architecture
Retrieval-based Systems
Security and Privacy Compliance
Computer Science
Verification Infrastructure

Summary

Our direct client, a global professional-services firm, is building autonomous AI agents for use in regulated financial work. They need a hands-on lead to design the independent verification layer. It sits outside the agent teams and decides whether an agent's behavior can be relied on. The client's position is that trust is the product: every result must be traceable to its evidence, and every limit must be documented.

You build the system that tries to break, measure, and certify the agents. You don't build the agents themselves.

Note: this work can be performed from home, anywhere in the US (most meetings will be on East Coast time). It is also open as a full-time salaried position as an alternative to long-term contract.
What you'll do

Design the "referee" system. Build repeatable, isolated test environments in which agents act against simulated tools that keep their own state. Wire the results into release pipelines so a failing score blocks a ship decision. The referee's own outputs (scores, evidence, verdicts) must be versioned and able to withstand outside scrutiny.
Decide what "correct" means for multi-step work. Static benchmarks aren't enough. You'll define how to judge:
- whether numbers and facts match known-true data, including a tolerance for what counts as a meaningful error rather than a trivial one
- the whole journey of a run: how it planned, which tools it chose, how it recovered from mistakes, and whether it wandered from its goal
- whether each conclusion can be followed back to the specific source that supports it
- whether behavior stays within accounting and regulatory rules
- whether the measurements themselves are statistically sound, including how many runs are enough, how to separate real regressions from noise, and the difference between "succeeds at least once in k tries" and "succeeds every time in k tries"
Lead the team. Set the technical direction, mentor engineers, and set the standard for the test infrastructure.
Build and calibrate model-based graders. When one AI grades another, you'll measure how often it agrees with human reviewers, check it for bias, and watch it for drift over time.
Treat test suites as products. That means sealed "never-trained-on" reference sets, manufactured hard cases, a written pass/fail agreement for each agent capability, and tripwires that catch quiet performance decay.
Connect lab results to real life. Show whether pre-release scores actually predict production behavior, and route production failures back into the test sets.
Attack the agents. Probe for hidden instructions smuggled in through tool outputs, privilege creep, data leakage, and agents gaming their own scoring.

Must have:

BS/MS in CS, engineering, statistics, or similar
7+ years of software engineering on complex, enterprise-grade systems
2+ years as a technical lead or architect
Strong Python, plus working experience with at least one framework for orchestrating LLM-based agents
You've shipped verification infrastructure for long-running, tool-using agents where the results actually decided release outcomes
Deep experience automating test or ML-operations pipelines at scale (containers, orchestration, continuous delivery)
Solid grasp of LLMs, retrieval-based systems, and agent architectures

Nice to have:

Regulated-industry background (audit, accounting, fintech, regtech)
Building systems where decisions must be explainable and auditable
Security and privacy compliance exposure
Testing systems with multiple cooperating agents, especially where handoffs fail
Hands-on use of public agent benchmarks or evaluation tooling (tell us which, and what you'd change)

Immediate interview and start - send resume today for immediate consideration!

By applying for this job, you agree to receive calls, AI-generated calls, text messages, or emails from Benchmark IT, LLC and its affiliates, and contracted partners. Frequency varies for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You can reply STOP to cancel and HELP for help. You can access our privacy policy here: bmarkits.com/privacy-policy

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 10208703
Position Id: 5612
Posted 11 hours ago

Company Info

About Benchmark IT- Technology Talent

Since 2007, Benchmark IT has delivered superior technology staffing solutions to employers throughout the New York metropolitan area and beyond. Our experienced recruiters consistently identify best-fit talent by combining the latest recruiting technology with seasoned insight and perspective.

Whether you’re an employer or a job seeker, we approach every recruiting assignment with keen precision, dedication to ethics, and personalized service. Our best-in-class recruiting process and high-touch approach produce outstanding, long-lasting placements.

Connecting the right talent with the right role at the right company creates success. Knowing how to put it all together is our expertise. Contact us today to get started.

Go to company profile

Fitment

Dice Job Match Score™

To see how well you match this job, please log in or create an account.

Once logged in, be sure to complete your profile to get the most accurate match score.

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Remote

•

Today

Senior AI Architect / EngineerU.S. based long-term contract opportunity | Primarily remote | Travel to New Jersey approximately every 1 2 months For our direct client, a prestigious global firm, we seek senior AI architects and engineers to build a high-priority application headed for production and help shape it into a scalable platform. This is a hands-on role with room to make meaningful architecture decisions. You ll write code, guide technical work across teams, and help establish practical

Easy Apply

Contract

Depends on Experience