Community project / Evaluations and independent research

Vev

Independent Jev-style decision server with vision input, fine-tuned from Qwen3.5 4B and 9B, that answers Choice, Score, and Noul questions about text, JSON, and images, with screenshots or photos placed directly in state, through a /v1/systemone-compatible server; the repository has Apache-2.0 code and per-set results on public text and image judgment sets, requests are processed on your own GPU after the weights download from Hugging Face, and the weights are CC BY-NC 4.0, for non-commercial use only.

Open the original resource ↗ See it in the full directory Start a reading list → Browse all Evaluations and independent research

Download share card ↓

This is an independent community listing. Check the source, license, data handling, and evaluation caveats before relying on a project. Inclusion is not an endorsement or security review.

Found an outdated or inaccurate detail? Report a correction →

Explore more in Evaluations and independent research

• 

  Von

  Open local Choice, Noul, and Score model with public weights, training code, and a System One-shaped API. Benchmark comparisons are author-reported: its README gives a 91.23% accuracy headline without a matching result artifact in the repo, while its 49-task table reports 71.5% macro accuracy.

  Read listing →
• 

  WebJev

  Apache-2.0 open-weight decision model for browser agents (a Qwen3.5-35B-A3B fine-tune) that answers Choice, Noul, and Score through a Jev-compatible /v1/systemone API and ships its training recipe, live-web training data, and a deterministic-verifier evaluation on 125 hand-picked real-website tasks. Its authors report 47/122 gradable successes versus 20/120 for Jev 1.13 inside the same agent, and 83.72% versus 84.70% mean accuracy on eight single-step typed-decision benchmarks; the live-web runs happened on different days and had different gradable subsets. It is a separate Jev-style alternative, not hosted Jev: self-hosting the weights needs one 80 GB GPU, and page observations go to the configured decision endpoint and text model.

  Read listing →
• 

  When a Judgment Layer’s Self-Reported Fields Lie

  Independent measurement of Jev’s typed decisions as one of three judgment layers run on identical items (Laya locally, Jev and a frontier model remotely), with per-call ledger-based cost accounting and Murphy/Brier/ECE calibration against binary ground truth; the Jev-specific findings are a verdict vocabulary reaching three values where six are documented, and a sufficient field that does not separate thin evidence from contradictory evidence, and the claim about contradiction rests on seven live readings that have no JSON artifact behind them, which the write-up states in its own errata rather than leaving to be discovered.

  Read listing →