Test the judgment.
Jev answers a yes/no, choice or score question in about 300 ms for a fraction of a cent. jevtool tells you whether those answers are right for your data: accuracy, the cases it misses, a confidence cutoff that is safe to automate at, and what changes on the next Jev version.
npx jevtool eval example:support-triage
Open source, MIT, zero dependencies. Bring your own TypeSafe key.
$ jevtool eval example:support-triage --compare jev-1.14.0 27 cases · PASSED jev-1.13.0 category choice 96% (26/27) [en 95% · zh 100%] escalate noul 100% (27/27) p50 272 ms · 553 tokens/call · ~$0.023 per 1k misses: s12 category: got refund, expected bug|billing (certainty 0.96) jev-1.14.0 category choice 96% (26/27) escalate noul 100% (27/27) jev-1.13.0 → jev-1.14.0: no answers changed
Ten judgments, run live.
Every suite ships with the CLI. Accuracy pools every question in the suite. "Automated at 95%" is the share of cases above the confidence cutoff where the automated answers stay at least 95% correct; the rest go to a person or a bigger model.
| Domain | Suite | Cases | Accuracy | Automated at 95% |
|---|---|---|---|---|
| Customer support | support-triageRoute emails to cancel, refund, billing, bug or how-to; flag chargeback threats | 27 | 98% | 100% |
| Chatbots | chat-inbound-guardPrompt injection, spam, and whether a message needs a reply | 21 | 90% | 76% |
| LLM apps | model-routingSmall, mid or frontier model for this request | 19 | 100% | 100% |
| AI answers | rag-fact-checkSource supports, contradicts or never mentions a claim | 16 | 94% | 88% |
| Data | entity-matchTwo messy records: same company, person or product? | 14 | 100% | 100% |
| E-commerce | review-moderationPublish, hide or escalate a product review | 14 | 100% | 100% |
| Personal | inbox-triageAction, security alert, receipt, FYI, newsletter or spam | 14 | 93% | 93% |
| Sales | lead-routingSales call, self-serve, nurture or disqualify | 12 | 92% | 75% |
| Legal | contract-riskAuto-renewal, uncapped liability, one-sided changes | 12 | 97% | 92% |
| Engineering | code-review-lintSwallowed errors, logged secrets, misleading commits | 12 | 97% | 92% |
jev-1.13.0, run 2026-10-19 with jevtool eval. Median latency 270–770 ms per case; about $0.02 per thousand calls at list price. Cases and labels are hand-written smoke tests of 12–27 cases each, not benchmarks. Automated share is the lowest across a suite's questions. Run any of them yourself: in your browser with your own key, or npx jevtool eval example:NAME, then swap in your own data.
One CLI, three jobs.
npm install -g jevtool
Accuracy per question and per tag, every miss, a suggested confidence cutoff, option-order checks, and which answers flip between Jev versions. Gate CI with --min-accuracy.
Check an agent's work against your checklist, the Git diff and freshly executed tests. A failing linked test always overrides the model.
Set up a checklist →Find records such as “payment succeeded but membership is missing.” Original text, line numbers, uncertain cases kept for review.
Filter your first file →Jev inside Claude Code.
Add the marketplace once, then install what you need.
claude plugin marketplace add jevtool/claude-plugins claude plugin install jev-guard@jevtool
Before Claude runs a command, writes a file or fetches a URL, Jev scores destructive, out-of-workspace, exfiltration, fit-to-task and severity. Blocks or asks. Never auto-approves, so your permission rules still apply.
How it decides →Claude turns your data into a labeled suite, runs it, reads the misses and tightens the question, then re-runs to make sure nothing regressed.
Read the skill →Before saying a task is finished, Claude checks the checklist, the diff and freshly run tests, and reports what still needs a look.
Read the skill →Ask one narrow question.
Three recorded answers. To run your own suite, open the online runner.
Free to run. Pay when you want it done for you.
Everything on this page: jevtool eval, verify, grep, the online runner, ten example suites and the three Claude Code plugins. Jev usage is billed by TypeSafe to your own key.
POST jevtool.dev/v1/decisions routes to Jev on TypeSafe or any model on OpenRouter's Decisions catalog, with automatic fallbacks. Your key is forwarded per request and never stored. Works with the TypeSafe SDKs.
We take one decision in your product, for example ticket routing, moderation, lead scoring or a risk flag, and deliver the questions, a labeled suite built from your data, a defensible confidence cutoff, the integration code and a CI gate. You keep everything.
Book a sprint →Saved suites, run history across Jev versions, shareable reports and team-wide jev-guard policies with an audit log. Built on your own TypeSafe key.
Tell me when it ships →Payments are handled by Stripe. Sprint scope and start date are confirmed by email before any charge.
Shipping a Jev feature?
We build it with you on your own data: the questions, a labeled test suite, a confidence cutoff you can defend, and a CI gate so model upgrades can't silently break it. Support triage, moderation, routing, risk flags, data matching.
Get new suites and plugins.
One short email when a new tool, plugin or Jev version comparison ships.