GenAI Evaluation Engineer / LLM

JOB DESCRIPTION;

We are seeking Ex apple candidate for Gen AI Engineer/LLMs with RAG;

We are seeking a highly analytical GenAI Evaluation Engineer to assess, validate, and improve the quality of Generative AI systems. This role is focused on systematically evaluating Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) pipelines, and agentic AI systems using a combination of automated metrics, human-in-the-loop review, and critical analysis. The ideal candidate will play a key role in ensuring AI outputs are accurate, reliable, safe, and aligned with business objectives.

Key Responsibilities

LLM & GenAI Evaluation

• Evaluate LLM responses for accuracy, relevance, reasoning quality, hallucinations, bias, and safety.

• Design and execute qualitative and quantitative evaluation frameworks for GenAI systems.

• Perform rubric-based human annotation and review of model outputs.

• Analyze failure modes and provide actionable feedback to engineering and research teams.

RAG & Agentic Systems

• Assess end-to-end performance of RAG pipelines, including retrieval quality, grounding, and context utilization.

• Evaluate multi-step agentic workflows for logical consistency, task completion, and tool usage.

• Identify breakdown points in reasoning chains and system orchestration.