Gen AI Engineer

Hybrid in Mountain View, CA, US • Posted 1 day ago • Updated 1 day ago

Contract W2

25% Travel Required

Hybrid

$40 - $60/hr

ThinkBeyond Inc

Fitment

Dice Job Match Score™

🎯 Assessing qualifications...

Job Details

Skills

A/B Testing
Analytical Skill
Artificial Intelligence
Conflict Resolution
Continuous Delivery
Continuous Integration
Control Flow
Databricks
Decision-making
Evaluation
FOCUS
Generative Artificial Intelligence (AI)
LangSmith
Management
Optimization
Orchestration
Problem Solving
Python
Recruiting
Workflow

Summary

Role Overview
GenAI Solutions Engineer focused on evaluation, observability, and performance measurement of GenAI systems.
Primary expectation:
→ Ability to quantify, evaluate, and improve system performance
Core Responsibilities
1. Evaluation & Metrics (Primary Focus)
Define and measure:
Accuracy (primary metric)
Precision, recall, confusion metrics
Hallucination rate
Faithfulness / grounding
Build ground truth datasets
Evaluate retrieval:
Precision@k
Recall@k
Perform:
Prompt A/B testing
Bias reduction
2. Evaluation Tooling
RAG / RAGAS
Databricks / MLflow
Arize, TruLens (production monitoring)
DeepEval (CI/CD integration)
LangSmith (enterprise)
LLM-as-judge frameworks
3. GenAI Workflow Orchestration
Control flow and logging
Memory management
Observability pipelines
Document comparison workflows

<image003.png>

4. Agentic AI Evaluation
Measure:
Tool correctness
Agent success rate
Use:
LLM-as-judge for agent evaluation

<image003.png>

5. Observability
Monitor:
Latency
Token/cost usage
Response quality
Hallucination trends
Retrieval accuracy

<image003.png>

6. System Thinking
Structured problem-solving
Metric-driven decision making
Clear articulation of tradeoffs:
Precision vs recall
Cost vs performance

<image003.png>

Engineering Requirements
Strong Python (data + evaluation focus)
Strong analytical thinking (more important than complex coding)

<image002.png>

Nice to Have
Cost optimization awareness
Platform tradeoff understanding

<image003.png>

Key Hiring Insight
Most candidates fail because they:
Lack depth in evaluation frameworks
Cannot clearly explain metrics or evaluation methodology

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91162606
Position Id: 8928659
Posted 1 day ago

Company Info

About ThinkBeyond Inc

Our team has a strong bond with the local community; they possess a profound understanding of the people and the market.

The Vision of Think Beyond is

Commitment to Excellence: Emphasize the consultancy's dedication to delivering high-quality recruitment services. Highlight the focus on finding top-notch candidates who possess the skills, experience, and cultural fit necessary to drive business success.

Strategic Partnership: Position the consultancy as a strategic partner for businesses, working closely with clients to understand their unique needs, goals, and company culture. Emphasize the importance of building long-term relationships and providing customized solutions.

Innovation and Adaptability: Highlight the consultancy's ability to stay ahead of emerging trends and technologies in the recruitment industry. Emphasize a commitment to continuous learning, leveraging innovative tools, and adopting new methodologies to ensure the best outcomes for clients.

Ethical and Professional Standards: Showcase the consultancy's commitment to operating with integrity, maintaining confidentiality, and adhering to the highest ethical standards in all aspects of the recruitment process. Highlight the importance of building trust and fostering positive relationships with both clients and candidates.

Go to company profile

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Gen AI Engineer

Mountain View, California

•

2d ago

Gen AI Engineer Location Mountain View, CA / Remote Mandatory Skills : GenAI Solutions Engineer focused onevaluation, observability, and performance measurementof GenAI systems.Ability toquantify, evaluate, and improve system performance Job Description Core Responsibilities 1. Evaluation & Metrics (Primary Focus) Define and measure: Accuracy (primary metric) Precision, recall, confusion metrics ,Hallucination rate Faithfulness / grounding Buildground truth datasets Evaluate retr

Easy Apply

Contract, Third Party

$60 - $70

Software Engineer (GenAI / RAG Systems)

Hybrid in Mountain View, California

•

14d ago

Role: Software Engineer (GenAI / RAG Systems) Location: Mountain View, CA Duration: 6 Months (Likely Extension) About the Role Join a high-impact engineering team working on productionizing internal AI tools used across Google. These tools have already proven value and are now being scaled into robust production systems. You ll operate as a Lead IC (Individual Contributor) owning architecture, system design, and execution for GenAI-powered applications. What You ll Do Build and scale RAG-based

Easy Apply

Contract

$90 - $110

Full Stack AI Engineer

Sunnyvale, California

•

14d ago

A globally leading consumer device company based in Sunnyvale, CA is looking for a Full Stack AI Engineer to join their team. Job Responsibilities: Serve as a Founding AI Engineer, partnering with internal and cross-functional teams to define AI product features, scope, and the platform s technical foundation. Design and implement generative AI systems, including multi-agent workflows, LLM orchestration patterns, tool-calling architectures, and MCP integrations to connect models with enterprise

Easy Apply

Contract

$69 - $84

AI QA Automation Engineer

Hybrid in Cupertino, California

•

2d ago

Role: AI QA Automation Engineer Location: Cupertino, CA OR Austin, TX (Hybrid T-W-T Onsite) Duration: 6 Months Job Description: QA Engineer focused on testing AI/ML systems and building automation frameworks to ensure model quality, reliability, and performance. Key Responsibilities Design and implement automated tests for AI/ML featuresValidate model outputs for accuracy, consistency, and edge casesBuild and maintain automation frameworks and evaluation pipelinesPerform data-driven testing for

Easy Apply

Contract

$55 - $70

Search all similar jobs