Community project / Evaluations and independent research

WebJev

Apache-2.0 open-weight decision model for browser agents (a Qwen3.5-35B-A3B fine-tune) that answers Choice, Noul, and Score through a Jev-compatible /v1/systemone API and ships its training recipe, live-web training data, and a deterministic-verifier evaluation on 125 hand-picked real-website tasks. Its authors report 47/122 gradable successes versus 20/120 for Jev 1.13 inside the same agent, and 83.72% versus 84.70% mean accuracy on eight single-step typed-decision benchmarks; the live-web runs happened on different days and had different gradable subsets. It is a separate Jev-style alternative, not hosted Jev: self-hosting the weights needs one 80 GB GPU, and page observations go to the configured decision endpoint and text model.

Open the original resource ↗ See it in the full directory Start a reading list → Browse all Evaluations and independent research

Download share card ↓

This is an independent community listing. Check the source, license, data handling, and evaluation caveats before relying on a project. Inclusion is not an endorsement or security review.

Found an outdated or inaccurate detail? Report a correction →

Explore more in Evaluations and independent research

• 

  When a Judgment Layer’s Self-Reported Fields Lie

  Independent measurement of Jev’s typed decisions as one of three judgment layers run on identical items (Laya locally, Jev and a frontier model remotely), with per-call ledger-based cost accounting and Murphy/Brier/ECE calibration against binary ground truth; the Jev-specific findings are a verdict vocabulary reaching three values where six are documented, and a sufficient field that does not separate thin evidence from contradictory evidence, and the claim about contradiction rests on seven live readings that have no JSON artifact behind them, which the write-up states in its own errata rather than leaving to be discovered.

  Read listing →
• 

  Awesome Jev Robustness

  Curated index of independent Jev tests grouped by calibration, consistency, perturbation, injection, abstention, failure modes, and language; it links original sources, reports their numbers as author findings, and keeps task benchmarks without a robustness angle in a separate data file.

  Read listing →
• 

  blink

  Self-hosted typed-decision models with Apache-2.0 code that return Choice, Noul, and Score option probabilities in one forward pass per batch through a TypeSafe-compatible API; the weights are non-commercial research and evaluation only, and there is no hosted TypeSafe-compatible endpoint.

  Read listing →