LLM Evaluation
Benchmark model versions against human preference
Run head-to-head comparisons between model versions, prompts, or configurations, and get preference rates with the disagreement made visible rather than averaged away.
What workers do
- Side-by-side preference between two model outputs
- Rate a response against a rubric you define
- Identify which model followed a system prompt
- Mark refusals that should not have been refusals
What you upload
- prompt
- response_a
- response_b
- system_prompt
Any format: CSV, TSV, JSON, JSONL, XLSX or TXT. Columns are detected automatically.
Common questions
- Can I use my own rubric?
- Yes. The task builder turns a description of your rubric into worker instructions and answer controls, which you can then edit directly.