AI Evaluation
Have humans judge what your model produced
Send model outputs to qualified people who read them and answer a specific question about each one. You get a structured answer per item with a confidence score.
What workers do
- Choose which of two responses better answers the prompt
- Rate correctness, relevance, completeness, clarity and safety
- Decide whether a response followed the instruction it was given
- Flag responses that are confidently wrong
What you upload
- prompt
- response
- response_a / response_b
- context
Any format: CSV, TSV, JSON, JSONL, XLSX or TXT. Columns are detected automatically.
Common questions
- How many people see each output?
- You choose — 1, 3, 5 or 7. Three is the default. More judgments raise confidence and cost proportionally.
- What happens when evaluators disagree?
- Items below your confidence threshold are automatically sent to additional workers. If they still do not converge, the item is flagged for review rather than reported as an answer.