Tymon Global
Tymon Global

Data Scientist Generative AI & RAG Evaluation

Hybrid in Mountain View, CA, US • Posted 2 days ago • Updated 15 hours ago
Contract W2
12 Months
No Travel Required
Able to Sponsor
Hybrid
Depends on Experience
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Generative Artificial Intelligence (AI)
  • LangChain
  • Machine Learning (ML)
  • Python
  • Data Analysis

Summary

We are hiring a Data Scientist with strong Applied AI experience to develop, test, and improve intelligent applications powered by Large Language Models. The role focuses on building reliable AI solutions, optimizing retrieval-based architectures, and establishing systematic evaluation approaches for LLM-generated outputs. Candidates should have strong Python and analytical capabilities, along with practical experience in model benchmarking, automated quality assessment, and human-assisted evaluation. Familiarity with modern AI orchestration frameworks and cloud-based ML platforms is expected.

Roles & Responsibilities

  • Develop and enhance intelligent applications using Large Language Models, advanced prompting strategies, Python-based data processing, and enterprise AI APIs.
  • Design retrieval-based AI solutions by implementing embedding generation, vector search, document indexing, context retrieval, and response optimization techniques.
  • Establish model assessment methodologies to measure the consistency, factual accuracy, relevance, and overall performance of GenAI applications.
  • Implement automated evaluation pipelines using LLM-as-a-Judge, human feedback mechanisms, and frameworks such as RAGAS, DeepEval, TruLens, and LangSmith.
  • Analyze model performance using quantitative measures, including precision, recall, F1-score, retrieval quality, groundedness, and response relevance.
  • Develop multi-step AI workflows and intelligent agent solutions using LangGraph, LangChain, and other orchestration technologies.
  • Perform exploratory data analysis, benchmarking, and evaluation reporting using Python, Pandas, SQL, Excel, and relevant ML experimentation tools.
  • Integrate AI monitoring, experiment tracking, evaluation logging, and model performance analysis using MLflow, Databricks, Arize AI, and AWS services.
 
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10483217
  • Position Id: 9136906
  • Posted 2 days ago
Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.
More jobs at Tymon Global in Mountain View, CA