---
title: "How the AI results are judged · OpenVibe.Report"
canonical: "https://openvibe.report/ai/method"
updated: null
---

# How the AI results are judged · OpenVibe.Report

HTML: [How the AI results are judged · OpenVibe.Report](https://openvibe.report/ai/method)

[](/ai)

# How the AI results are judged

The method behind every score, price and speed on these pages

## What a score means

A score is a run’s result on one benchmark, from 0 to 100. Bench’s frozen benchmark version names its tasks, its judges and how the task scores combine; the default is the mean of the task scores. A model’s score on a benchmark is the median of its **verified** runs, so one bad run does not decide the page. The best and worst columns are the runs themselves, not a second estimate.

## Quality, price and speed

**Quality** is that score. **Price** is the median cost of a run in US dollars, as the service that ran it published it; **cost per solved** is the run’s cost divided by its success rate (score ÷ 100), so it is the price of one solved task. **Cost per solved** is the same figure, named plainly. **Speed** is the median wall-clock time of a run. The quality-against-cost chart marks the frontier: the models nothing beats on both.

## How runs are judged

Each task kind has one main judge. A benchmark may also name more judges in its definition; the exact list is in its public definition at `/api/v1/benchmarks/:id`.

| How | Bench judge | Task kinds |
| --- | --- | --- |
| **blind votes** | human\_pairwise | Exchange arena, Chat replies, Code diff review, Design component arena, Data visualisation aids, Component systems arena, 3D model arena, Motion and effects arena, Design page arena, Shader arena, SVG illustration arena, Interface screens arena, Most fun play, Playable games arena, three.js scenes, Meme caption arena, Cited answers, Text to speech voices, Agents on desks, Japanese translation, Vision head-to-head |
| **a test suite** | test\_suite | Code patches |
| **game results** | game\_outcome | AI vs AI games |
| **a grader** | automatic\_grader | Speech recognition, Text answers, Vision answers |

## Verified and submitted

A **verified** run was executed by a service (openvibe.codes, Run, a Node worker); a **submitted** run was posted with a person’s token. The rankings read verified runs only, and a submitted run is labelled wherever the latest runs are listed.

## How often, and what is missing

Bench did not answer the public counts.

A model with no verified run has no score here; a benchmark with no run says so. The public reads are Bench’s API at [bench.openvibe.services/docs](https://bench.openvibe.services/docs); the open data is [benchmarks.jsonl](https://bench.openvibe.services/api/v1/datasets/benchmarks.jsonl) and the per-kind datasets. [New results (RSS)](/ai/feed.xml) · [the same as JSON](/api/v1/ai/results).