RAG Evaluation
Check whether answers are actually supported by their sources
Retrieval systems fail quietly: the answer sounds right and the citation is wrong. Have people read the source and the answer together, and say whether one supports the other.
What workers do
- Judge whether an answer is supported, partially supported, or unsupported
- Identify which specific claim is not in the source
- Rate retrieval relevance for a query
- Flag answers that contradict their own citation
What you upload
- question
- source
- ai_answer
Any format: CSV, TSV, JSON, JSONL, XLSX or TXT. Columns are detected automatically.
Common questions
- Do evaluators use outside knowledge?
- No — the instructions state explicitly that an answer can be true in the world and still unsupported by the source, and that only the source counts.