LLM Evaluation jobs in Virginia

Principal/Staff Data Scientist, GenAI/LLM Evaluation, Long-Term Contract

Remote

•

2d ago

Principal/Staff Data Scientist, GenAI/LLM Evaluation Our direct client, a prestigious global firm, is creating a new advanced-technology organization. They are standing up AI evaluation as a first-class function and need a genuinely senior leader to build it. The firms products and agents will do various business functions which must be auditable. The standard is correct results every time, not good enough for a personal assistant. Because large language models are probabilistic, this takes re

Easy Apply

Contract

Depends on Experience

Senior AI/LLM Optimization Engineer

Hybrid in McLean, Virginia

•

Today

Data Capital Inc. is seeking a highly experienced AI/LLM Optimization Engineer to optimize production-grade Generative AI applications for accuracy, performance, scalability, and reliability. This is a hands-on engineering rolenot a prompt engineering or POC-focused position. Key ResponsibilitiesOptimize enterprise LLM applications, RAG pipelines, and AI agents. Improve model accuracy, response quality, throughput, and inference latency. Implement advanced RAG techniques, embeddings, hybrid sear

Easy Apply

Full-time

Depends on Experience

Senior Software Engineer, AI Evaluation (Python / TypeScript / Go), Remote Contract

Remote

•

Yesterday

Remote contract work for senior engineers: design rubrics and evaluation criteria for coding AI, review and label model output, and build data generation and evaluation pipelines. 5+ years hands-on Python, TypeScript, Go or JavaScript- Strong background building and maintaining pipelines and infrastructure- Rubric or preference-data experience is a plus- 20 to 40 hours a week with 4 hours overlapping US Pacific time; 3-month contract, immediate start- Top-of-market pay: up to $90+/hr for the mo

Contract

Depends on Experience

AI Automation Engineer

Hybrid in Vienna, Virginia

•

18d ago

Job Title: AI Test Engineer Location: Vienna, VA Type: Contract Work Model: Hybrid onsite and remote Hours: 40.0 Responsibilities Help design, develop, and maintain AI testing and evaluation frameworks for generative AI, predictive AI, and AI-powered developer experience (DevEx) solutions. Create and execute automated test suites that validate AI model behavior, output quality, accuracy, reliability, and performance. Develop automated evaluation pipelines that integrate with CI/CD workflows

Easy Apply

Contract

Depends on Experience

Senior Data Scientist / Data Engineer Generative AI Engine Optimization

Remote

•

Yesterday

Senior Data Scientist / Data Engineer Generative AI Engine Optimization Job Summary We are seeking a Senior Data Scientist / Data Engineer to own the complete data pipeline infrastructure, metric governance, warehouse architecture, executive dashboards, and AI-access layer for our Generative Engine Optimization (GEO) initiative. As consumer discovery shifts toward generative AI environments such as ChatGPT, Google AI Overviews & AI Mode, Gemini, Claude, and Perplexity this role ensures our core

Easy Apply

Contract

$80 - $95

EOIR Program Manager - AI TESTING

Alexandria, Virginia

•

30+d ago

Job Title: EOIR Program Manager - AI TESTING Location: Alexandria, VA Type: Contract Compensation: $150,000.00-160,000.00 USD Work Model: Onsite onsite Hours: 40.0 Security Clearance: Must be able to obtain and maintain a DOJ Public Trust level security clearance (candidates with an active or recently active Tier 2 or higher clearance are strongly preferred). Responsibilities Plan, initiate, and manage all IT/AI project activities across SOW Task Areas 1-5, monitoring progress to ensure

Easy Apply

Contract

$200000

Member of Technical Staff, Coding Research

Remote or New York, New York

•

Today

Job Title: Member of Technical Staff, Coding Research Job Type: Full-time Location: Remote The Role We are seeking a Member of Technical Staff to help advance the evaluation and development of frontier coding agents. Sitting at the intersection of AI research, software engineering, and model evaluation, you will design the benchmarks, methodologies, and data systems that shape how next-generation coding models are measured and improved. What You'll Do Design and own evaluation frameworks fo

Easy Apply

Full-time

$$600K - $700k/yr

AI Scrum Master with Python - 100% remote

Remote

•

17d ago

Job Title:AI Scrum Master with Python skills MUST HAVE PYTHON PROGRAMMING SKILLS Location:100 % remote Type: Contract To Hire Compensation:W-2 Negotiable Work Model:Remote Hours: 40 hours a week Responsibilities Serve as SAFe Agile Scrum Master for one or more Agile teams delivering AI-enabled information systems in a SAFe Agile environment. Facilitate Agile ceremonies including daily stand-ups, sprint planning, backlog refinement, sprint reviews, retrospectives, and other team synchronization

Easy Apply

Contract

Depends on Experience

Remote Business Analyst - US

Remote

•

2d ago

About Turing: Turing is one of the worlds fastest-growing AI companies accelerating the advancement and deployment of powerful AI systems. Turing helps customers in two ways: Working with the worlds leading AI labs to advance frontier model capabilities in thinking, reasoning, coding, agentic behavior, multimodality, multilinguality, STEM and frontier knowledge; and leveraging that work to build real-world AI systems that solve mission-critical priorities for companies. Role Overview: We are lo

Easy Apply

Contract

15 - 20

Remote Business Analyst - US

Remote

•

2d ago

Easy Apply

Contract

15 - 20

Medical Experts - Dermatology (USA)

Remote

•

24d ago

Job Title: Medical Experts - Dermatology (USA) Experience: 5 Years to 5 Years Job Type: Remote Location: N/A Openings: 40 Currency: USD Visa: // H1B / EAD Driver's License: Required Valid US DL Job Description: Years of Experience: A minimum of 5 years of post-residency experience practising as a dermatologist About Turing: Based in San Francisco, California, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced

Easy Apply

Contract

Depends on Experience

AI Engineer - Senior

Chantilly, Virginia

•

Today

At Accenture Federal Services, nothing matters more than helping the US federal government make the nation stronger and safer and life better for people. Our 13,000+ people are united in a shared purpose to pursue the limitless potential of technology and ingenuity for clients across defense, national security, public safety, civilian, and military health organizations. Join Accenture Federal Services, a technology company within global Accenture. Recognized as a Glassdoor Top 100 Best Place to

Full-time

USD 94,000.00 - 187,600.00 per year

Software Engineer AI Evaluation & Automation

Remote

•

4d ago

Software Engineer AI Evaluation & Automation Remote 12 Months contract Role Overview Help build and scale the tooling we use to measure how well AI-powered software development tools actually perform. You'll develop evaluation harnesses, automate benchmark runs, and help make sure the results we produce are reproducible and hold up to scrutiny. This is an engineering role, but a lot of the work is about getting the measurement right, not just automating it. Key Responsibilities Build and integ

Easy Apply

Third Party, Contract

Depends on Experience

Product Engineer, AI Email Applications, Artificial Intelligence (AI) Required - Work From Home

Remote or San Francisco, California

•

Today

Company Description Job Description Product Engineer, AI Email Applications, Artificial Intelligence (AI) Required - Work From Home As Product Engineer, AI Email Applications, you will work at the intersection of user needs, model capability, and engineering constraints. The Product Engineer will work directly with Machine Learning (ML) and engineering teams to define how the system should behave, how we measure it, and how we continuously improve it. This is a deeply technical, hands-on role

Full-time

Senior Director, Frontier AI Research, Bay Area, NY

Remote or San Francisco, California

•

Today

EPAM's new Frontier AI business unit partners directly with leading AI labs and advanced AI organizations, translating their research and post-training objectives into technically rigorous, deliverable programs across evaluations, RL environments, and training data. The Senior Director, Frontier AI Research is EPAM's first dedicated Frontier AI research hire, operating across a broad set of capabilities rather than a single research domain. The role sits at the intersection of research, solution

Full-time

AI/LLM Developer

Arlington, Virginia

•

Today

Residency All applicants must currently reside in the United States Overview BigBear.ai is seeking a Mid-Level AI/LLM Developer to support the development and implementation of AI-focused initiatives and pilot efforts supporting the Transportation Security Administration (TSA). This role will contribute to the development of AI-enabled capabilities from concept through implementation and will work closely with technical and functional teams to design, develop, test, integrate, and deploy solut

Full-time

Software Engineer - Python/Typescript

Remote

•

2d ago

About TuringTuring is one of the worlds leading AGI infrastructure companies, working with frontier AI labs to accelerate model development through high-quality training data, evaluations, and engineering talent. About the RoleWere looking for experienced, hands-on software engineers to help evaluate and improve AI coding models. Rather than primarily building production applications, youll work with coding agents across real-world repositories and assess the quality of their work. Youll review

Easy Apply

Contract

200 - 300

AI Manager - Language Models (US Remote)

Remote or Illinois

•

Today

Company Overview At Motorola Solutions, we believe that everything starts with our people. We're a global close-knit community, united by the relentless pursuit to help keep people safer everywhere. We build and connect technologies to help protect people, property and places. Our solutions foster the collaboration that's critical for safer communities, safer schools, safer hospitals, safer businesses, and ultimately, safer nations. Connect with a career that matters, and help us build a safer

Full-time

USD 240,000.00 per year

Reverse Engineer/Software Tester

Arlington, Virginia

•

Today

Overview Systems Planning and Analysis, Inc. (SPA) delivers high-impact, technical solutions to complex national security issues. With over 50 years of business expertise and consistent growth, we are known for continuous innovation for our government customers, in both the US and abroad. Our exceptionally talented team is highly collaborative in spirit and practice, producing Results that Matter. Come work with the best! We offer opportunity, unique challenges, and clear-sighted commitment to

Full-time

USD 150,000.00 - 250,000.00 per year

Artificial Intelligence Engineer (Multiple Levels)

McLean, Virginia

•

Today

Responsibilities Noblis is seeking an Artificial Intelligence (AI) Engineer to work in dynamic mission-oriented environment within McLean, Virginia. As an AI Engineer you will work with mission users to understand requirements, and partner with data engineers to transform complex datasets into actionable insights, automate analyst workflows, and deliver scalable, reliable AI capabilities aligned with mission objectives. Job Responsibilities: Collaborate directly with mission users and stakehol

Full-time

USD 90,700.00 - 141,775.00 per year

LLM Evaluation jobs in Virginia, USA