Gen AI Engineer
Location – Mountain View, CA / Remote
Mandatory Skills : GenAI Solutions Engineer focused on evaluation, observability, and performance measurement of GenAI systems. Ability to quantify, evaluate, and improve system performance
Job Description –
Core Responsibilities
1. Evaluation & Metrics (Primary Focus)
- Accuracy (primary metric)
- Precision, recall, confusion metrics ,Hallucination rate
- Faithfulness / grounding
- Build ground truth datasets
- Evaluate retrieval:
- Perform:
- Prompt A/B testing
- Bias reduction
2. Evaluation Tooling
- Arize, TruLens (production monitoring)
- DeepEval (CI/CD integration)
3. GenAI Workflow Orchestration
- Document comparison workflows
4. Agentic AI Evaluation
- Agent success rate
- LLM-as-judge for agent evaluation
5. Observability
6. System Thinking
- Structured problem-solving
- Metric-driven decision making
- Clear articulation of tradeoffs:
Engineering Requirements
- Strong Python (data + evaluation focus)
- Strong analytical thinking (more important than complex coding)
Tekshapers is an equal opportunity employer and will consider all applications without regards to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law.