Content Evaluator/Reviewer

Remote • Posted 3 days ago • Updated 3 days ago
Contract W2
6 Months
No Travel Required
Remote
$42 - $44/hr
Fitment

Dice Job Match Score™

🛠️ Calibrating flux capacitors...

Job Details

Skills

  • Elasticsearch
  • Relevance Evaluator
  • AI Evaluator
  • Content Reviewer
  • Search Engine Evaluator
  • Search Evaluator
  • KQL
  • Kibana Query Language
  • relevance rater

Summary

About the role:

  • As a relevance evaluator you will be responsible for evaluating how relevance a given document (or image, video) is for a particular user query.
  • You will have to rate the document relevance 1-5 (from not relevant at all to exactly what the user was looking for).
  • Rating relevance requires first trying to understand the user intent, for example, where they researching a topic, or they were looking for their own document, and forming an opinion what would be a good document (or image, etc.) for this particular query for this particular user.
  • Other similar tasks involve evaluating quality of assistant answers, including various aspects of the answer: is it correct (the information answers the user intent correctly), is it descriptive, or is the system apologizing for not knowing the answer. Understanding correctness also requires some research on the terminology used.

 

Responsibilities:

  • Responsibility includes reviewing the query and document pairs, including any context provided by the system (such as when the query was issued and who did it), researching the problem trying to understand the user intent by using the product (search, including formulating queries for elastic search index in KQL), and then rating the document or other required artifact (e.g. assistant answer) according to the provided relevance rubric.

 

Requirements:

  • Understanding of English
  • Ability to formulate search or chatgpt-like queries

 

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 90989446
  • Position Id: 9032311
  • Posted 3 days ago
Contact the job poster
Ajit Maurya

Ajit Maurya

Amiseq Inc. Recruiter @ Amiseq Inc.
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Remote

9d ago

Easy Apply

Contract

Depends on Experience

Remote

Today

Full-time

USD 194,000.00 - 204,500.00 per year

Remote or Draper, Utah

Today

Contract

$40 - $50 hourly

Remote

Today

Easy Apply

Contract

Depends on Experience

Search all similar jobs