Community project / Evaluations and independent research

When a Judgment Layer’s Self-Reported Fields Lie

Independent measurement of Jev’s typed decisions as one of three judgment layers run on identical items (Laya locally, Jev and a frontier model remotely), with per-call ledger-based cost accounting and Murphy/Brier/ECE calibration against binary ground truth; the Jev-specific findings are a verdict vocabulary reaching three values where six are documented, and a sufficient field that does not separate thin evidence from contradictory evidence, and the claim about contradiction rests on seven live readings that have no JSON artifact behind them, which the write-up states in its own errata rather than leaving to be discovered.

Open the original resource ↗ See it in the full directory Start a reading list → Browse all Evaluations and independent research

Download share card ↓

This is an independent community listing. Check the source, license, data handling, and evaluation caveats before relying on a project. Inclusion is not an endorsement or security review.

Found an outdated or inaccurate detail? Report a correction →

Explore more in Evaluations and independent research

• 

  Awesome Jev Robustness

  Curated index of independent Jev tests grouped by calibration, consistency, perturbation, injection, abstention, failure modes, and language; it links original sources, reports their numbers as author findings, and keeps task benchmarks without a robustness angle in a separate data file.

  Read listing →
• 

  Bosun v3.1

  Apache-2.0 open-weight Qwen3 decision model family with released 0.6B and 1.7B weights and shipped choice, score, and noul inference; a separate Jev-style alternative, not hosted Jev.

  Read listing →
• 

  DecisionBench

  Open benchmark runtime for document-grounded decision models, with pinned tasks and a native OpenRouter adapter for Jev.

  Read listing →