Gen AI Engineer

Mountain View, CA, US • Posted 2 days ago • Updated 2 days ago
Contract Independent
Contract Corp To Corp
Contract W2
No Travel Required
On-site
$60 - $70/hr
Fitment

Dice Job Match Score™

📊 Calculating match score...

Job Details

Skills

  • LLM
  • RAG
  • Observability
  • Open AI

Summary

Gen AI Engineer

Location – Mountain View, CA / Remote 

 

Mandatory Skills : GenAI Solutions Engineer focused onevaluation, observability, and performance measurementof GenAI systems. Ability toquantify, evaluate, and improve system performance 

Job Description –

Core Responsibilities 

1. Evaluation & Metrics (Primary Focus) 

  • Define and measure: 
  • Accuracy (primary metric) 
  • Precision, recall, confusion metrics ,Hallucination rate 
  • Faithfulness / grounding 
  • Build ground truth datasets 
  • Evaluate retrieval: 
  • Perform: 
  • Prompt A/B testing 
  • Bias reduction 

 

2. Evaluation Tooling 

  • RAG / RAGAS 
  • Databricks / MLflow 
  • Arize, TruLens (production monitoring) 
  • DeepEval (CI/CD integration) 
  • LangSmith (enterprise) 
  • LLM-as-judge frameworks 

3. GenAI Workflow Orchestration 

  • Control flow and logging 
  • Memory management 
  • Observability pipelines 
  • Document comparison workflows 

 

4. Agentic AI Evaluation 

  • Measure: 
  • Tool correctness 
  • Agent success rate 
  • LLM-as-judge for agent evaluation 

5. Observability 

  • Monitor: 
  • Latency 
  • Token/cost usage 
  • Response quality 
  • Hallucination trends 
  • Retrieval accuracy 

6. System Thinking 

  • Structured problem-solving 
  • Metric-driven decision making 
  • Clear articulation of tradeoffs: 
  • Precision vs recall 
  • Cost vs performance 

Engineering Requirements 

  • Strong Python (data + evaluation focus) 
  • Strong analytical thinking (more important than complex coding) 

 

Tekshapers is an equal opportunity employer and will consider all applications without regards to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law.

 

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10335759
  • Position Id: 27079-17456-
  • Posted 2 days ago
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Hybrid in Mountain View, California

Yesterday

Easy Apply

Contract

40 - 60

Sunnyvale, California

14d ago

Easy Apply

Contract

$69 - $84

San Jose, California

Today

Easy Apply

Contract, Third Party

Depends on Experience

Santa Clara, California

6d ago

Easy Apply

Contract

Depends on Experience

Search all similar jobs