Which decision model should you trust?
Jev started the System One category; now OpenRouter lists dozens of models that take the same request. We run every one of them against the same labeled suites each week and record accuracy, how much you could automate, latency and the billed cost.
| # | Model | Mean accuracy | Automated at 95% | Median latency | Cost / 1k calls |
|---|---|---|---|---|---|
| Loading results… | |||||
By suite
| # | Model | Accuracy | Automated at 95% |
|---|
Not ranked
Read this before quoting a number
Run it on your data
The leaderboard tells you which models are worth trying. Only your own labeled cases tell you which one to ship.
export OPENROUTER_API_KEY=... npx jevtool eval my-suite.json --provider openrouter \ --model typesafe/jev-1.13 --compare inception/mercury-decide,cloudflare/clef-flash
Method, raw results per run, and how to contribute a suite: github.com/jevtool/benchmarks. Not affiliated with TypeSafe, OpenRouter or any model publisher.