Evaluation jobs in alameda, ca

Remote

•

Today

Join the Enterprise Agent Development Platform project at EPAM. We are building a cloud-native platform that enables engineering teams to develop, deploy, and operate AI agents in production. The platform combines modern agent frameworks, AWS infrastructure, CI/CD, observability, and governance to make AI development faster and more reliable. As a Python AI Evaluation Engineer , you will own a key part of this platform: building the capabilities that measure and validate the quality of AI soluti

Full-time

Principal/Staff Data Scientist, GenAI/LLM Evaluation, Long-Term Contract

Remote

•

Today

Principal/Staff Data Scientist, GenAI/LLM Evaluation Our direct client, a prestigious global firm, is creating a new advanced-technology organization. They are standing up AI evaluation as a first-class function and need a genuinely senior leader to build it. The firms products and agents will do various business functions which must be auditable. The standard is correct results every time, not good enough for a personal assistant. Because large language models are probabilistic, this takes re

Easy Apply

Contract

Depends on Experience

AI voice Evaluation Specialist

Remote

•

7d ago

Title: AI voice Evaluation Specialist Location: Remote - USA Duration: 4+ Months Project - W2 Regular hours Good Quality Headphone Is Must And Native English Speakers Only. A heavyaccent will not work. Previous experience is Speech Annotation or Audio engineering, or Podcasting. Job Description Scope of the Role: We are looking for detail-oriented Voice Specialists to evaluate the performance of AI models through real-time, voice-based conversations. In this role, you will interact

Easy Apply

Third Party, Contract

Depends on Experience

Software Engineer AI Evaluation & Automation

Remote

•

2d ago

Software Engineer AI Evaluation & Automation Remote 12 Months contract Role Overview Help build and scale the tooling we use to measure how well AI-powered software development tools actually perform. You'll develop evaluation harnesses, automate benchmark runs, and help make sure the results we produce are reproducible and hold up to scrutiny. This is an engineering role, but a lot of the work is about getting the measurement right, not just automating it. Key Responsibilities Build and integ

Easy Apply

Third Party, Contract

Depends on Experience

AI Evaluation Infrastructure Engineer

San Francisco, California

•

19d ago

It all started with an idea at Block in 2013. Initially built to take the pain out of peer-to-peer payments, Cash App has gone from a simple product with a single purpose to a dynamic ecosystem, developing unique financial products, including Afterpay/Clearpay, to provide a better way to send, spend, invest, borrow and save to our 50+ million monthly active customers. We want to redefine the world's relationship with money to make it more relatable, instantly available, and universally accessibl

Full-time

USD 263,600.00 - 395,400.00 per year

Software Engineer, Simulation & Evaluation

Alameda, California

•

Today

THE COMPANY Saildrone is a maritime defense company and the global leader in autonomous unmanned surface vehicles (USVs). With more ocean miles and real-world operational experience than any USV manufacturer or operator, Saildrone maintains active, combat-deployed systems supporting national security and force projection around the world, 24/7/365. Saildrone's manufacturing and R&D headquarters are located in Alameda, CA, with business development and sales operations in Washington, DC, and dep

Full-time

USD 144,500.00 - 300,000.00 per year

Senior Machine Learning Engineer, AI-Native Evaluation & Search

Remote or San Francisco, California

•

Today

Overview Atlassian is looking for a Senior Machine Learning Engineer to join our Search & Intelligence organization. We build AI-native experiences, agentic systems, models, evaluation frameworks, and data platforms that power Atlassian's AI products. Working at Atlassian Atlassians can choose where they work - whether in an office, from home, or a combination of the two. That way, Atlassians have more control over supporting their family, personal goals, and other priorities. We can hire peo

Full-time

USD 206,100.00 - 269,075.00 per year

Software Engineer - Generative Data Platform, Evaluation

San Francisco, California

•

Today

Apple's machine learning features are only as good as the data behind them, and our team builds the platform that lets teams create data that fills the gaps real-world collection can't, provides privacy-preserving ways to train and evaluate our models safely, and helps us ensure our products have seen diverse inputs to generalize properly across the many parts of the world where we operate. In the AI & Machine Learning (AIML) organization, our team builds the self-service tools and platform tha

Full-time

Applied AI Researcher, Agent Systems & Evaluation

Mountain View, California

•

Today

Who We Are Nuro is a self-driving technology company on a mission to make autonomy accessible to all. Founded in 2016, Nuro is building the world's most scalable driver, combining cutting-edge AI with automotive-grade hardware. Nuro licenses its core technology, the Nuro Driver , to support a wide range of applications, from robotaxis and commercial fleets to personally owned vehicles. With technology proven over years of self-driving deployments, Nuro gives automakers and mobility platforms a c

Full-time

USD 193,930.00 - 352,290.00 per year

Senior Software Engineer, Evaluation Engine

Remote

•

Today

At Vanta, our mission is to help businesses earn and prove trust.We believe that security should be monitored and verified continuously, and we empower companies to practice better security and prove it with ease. Vanta has a kind and talented team, and while some have prior security experience, many have been successful at Vanta without it. Our Senior Software Engineers independently drive complex technical work, shape the systems and technical decisions within their teams, and enable other eng

Full-time

USD 195,000.00 - 229,000.00 per year

Staff Software Engineer, Autonomy Evaluation

Remote

•

Today

Job Description General Motors is a global leader in advanced driver assistance. With Super Cruise hands-free technology in more than 500,000 Super Cruise-equipped vehicles on the road, and over 700 million hands-free miles driven, GM is proving that automation can be trusted, intuitive, and helpful. GM has the global reach to bring cutting-edge advances to everyday drivers at unprecedented scale. Join us to help deliver the next generation of safe and delightful personal autonomous vehicle exp

Full-time

Software Engineer - Evaluation

Fremont, California

•

Today

Founded in 2016 in Silicon Valley, Pony.ai has quickly become a global leader in autonomous mobility and is a pioneer in extending autonomous mobility technologies and services at a rapidly expanding footprint of sites around the world. Operating Robotaxi, Robotruck and Personally Owned Vehicles (POV) business units, Pony.ai is an industry leader in the commercialization of autonomous driving and is committed to developing the safest autonomous driving capabilities on a global scale. Pony.ai's l

Full-time

USD 120,000.00 - 200,000.00 per year

Senior Software Engineer, Autonomy Evaluation

Remote

•

Today

Job Description General Motors is a global leader in advanced driver assistance. With Super Cruise hands-free technology in more than 500,000 Super Cruise-equipped vehicles on the road and over 700 million hands-free miles driven, GM is proving that automation can be trusted, intuitive, and helpful. GM has the global reach to bring cutting-edge advances to everyday drivers at unprecedented scale. Join us to help deliver the next generation of safe and delightful personal autonomous vehicle expe

Full-time

Sr. Software Engineer I, Evaluation Services (Scala)

Remote or Chicago, Illinois

•

Today

Disclaimer: Please be advised that the most accurate and up-to-date information about our open roles-including job descriptions, compensation, and benefits-can only be guaranteed on our official job board. For the latest listings and details, please visit: ;br> JOIN US ON OUR MISSION TO BECOME THE #1 RETAIL TRADING PLATFORM IN THE WORLD Welcome to the dynamic world of NinjaTrader! As an industry-leading trading platform and futures broker, we're empowering traders to take control of their finan

Full-time

USD 125,000.00 - 175,000.00 per year

Software Engineer, Agent Platform

San Francisco, California

•

Today

WHY WE'RE LOOKING FOR YOU We're building AI-native products where model behavior is part of the product, not just an implementation detail. As LLMs become more capable, the bottleneck is no longer access to models-it's making non-deterministic systems reliable, evaluable, and trustworthy in production. We're hiring AI Engineers to own that problem end to end. This is not a role for someone who simply integrates AI APIs into features. It's for engineers who take responsibility for how probabili

Full-time

USD 163,800.00 - 306,000.00 per year

Member of Technical Staff (AI Software Engineer, Agents)

San Francisco, California

•

Today

Perplexity is seeking energetic engineers to join our highly driven Agents engineering team. The Agents team consists of backend, full-stack, and AI/ML engineers who collaborate to build harnesses and AI systems powering delightful agentic experiences. These experiences include Perplexity Computer (our platform for generalized frontier intelligence), the Comet ecosystem, our Agent API, and more. Our vision is to empower our users with agents that can faithfully actualize their intent, however a

Full-time

USD 220,000.00 - 405,000.00 per year

Machine Learning (ML) Engineer, Ranking & Retrieval, Machine Learning (ML) and Artificial Intelligence (AI) Required, Work From Home

Remote or San Francisco, California

•

Today

Company Description Job Description Machine Learning (ML) Engineer, Ranking & Retrieval, Machine Learning (ML) and Artificial Intelligence (AI) Required, Work From Home The Machine Learning (ML) Engineer will build and improve Machine Learning (ML) systems that power our AI-native applications. You will work on classification, retrieval, ranking, personalization and evaluation, taking ownership from model development to production deployment. This role is for engineers who enjoy solving pract

Full-time

Applied Machine Learning (ML) Engineer, Machine Learning (ML) and Artificial Intelligence (AI) Required, Work From Home

Remote or San Francisco, California

•

Today

Company Description Job Description Applied Machine Learning (ML) Engineer, Machine Learning (ML) and Artificial Intelligence (AI) Required, Work From Home The Applied Machine Learning (ML) Engineer will build and improve Machine Learning (ML) systems that power our AI-native applications. You will work on classification, retrieval, ranking, personalization and evaluation, taking ownership from model development to production deployment. This role is for engineers who enjoy solving practical

Full-time

Product Engineer, AI Email Applications, Artificial Intelligence (AI) Required - Work From Home

Remote or San Francisco, California

•

Today

Company Description Job Description Product Engineer, AI Email Applications, Artificial Intelligence (AI) Required - Work From Home As Product Engineer, AI Email Applications, you will work at the intersection of user needs, model capability, and engineering constraints. The Product Engineer will work directly with Machine Learning (ML) and engineering teams to define how the system should behave, how we measure it, and how we continuously improve it. This is a deeply technical, hands-on role

Full-time

Software Engineer, Workforce Operations

San Francisco, California

•

Today

About Assembled Assembled helps companies scale great customer support. As consumers, we've all been there. You need help, you're stuck on hold or going in circles with an AI, and your problem still isn't solved. Behind great support is a complex operational problem: thousands of people and AI agents working across teams, time zones, and products as new problems emerge. Assembled gives support leaders the tools to manage that complexity, from AI agents that resolve customer issues to software

Full-time

USD 180,000.00 - 230,000.00 per year

Evaluation jobs in Alameda, CA, USA