Pricing

Free to write and judge. Pay only to run.

Upvoted tests run on the roster for free. To run any test on any model right now, buy credits: provider cost plus 30%, quoted to the cent before anything runs.

Free

$0

For everyone.

  • Write public tests
  • Vote and judge blind
  • Free roster run at 3 upvotes
  • Leaderboards, charts, sharing
Create an account

Credits

from $10

Pay as you go. Never expire.

1 credit = 1 US cent of runs. Minimum $0.02 per model per run.

What a run costs

Running a pelican riding a bicycle on all 11 roster models costs $0.28 right now: from $0.02 on the cheapest model to $0.05 on the most expensive. Prices move with the providers' list prices, refreshed hourly.

  • You see the price first. Pick models and samples, and the run page shows every model's price and the total before you confirm.
  • Never more than quoted. If a run costs us less than expected, you pay less. If it costs more, we eat the difference.
  • Prepaid only. Runs draw from your balance; nothing is billed afterwards and a balance never goes negative.
  • No surprise refunds or cancellations. We only refund or cancel when you ask. Write to us any time.
Running evaluations for a lab? Sealed holdout sets are licensed separately. For labs

Tell us what broke, or what is missing