Community project / Evaluations and independent research

TypeSafe AI Benchmark

Side-by-side Jev and Qwen-on-Cerebras comparison with raw exports, cost accounting, methodology, and task-specific limitations.

Open the original resource ↗ See it in the full directory Start a reading list → Browse all Evaluations and independent research

Download share card ↓

This is an independent community listing. Check the source, license, data handling, and evaluation caveats before relying on a project. Inclusion is not an endorsement or security review.

Found an outdated or inaccurate detail? Report a correction →

Explore more in Evaluations and independent research

• 

  Verdict

  Apache-2.0 118M multilingual bi-encoder for Choice, Score, and Noul on a System One-compatible endpoint, with temperature scaling and optional split-conformal abstention whose coverage depends on its calibration assumptions; it can run locally on CPU or in a browser through ONNX, and the author's reported calibration and typed-decision results are task-specific, including a lower score than Laya on the published typed-decision suite.

  Read listing →
• 

  Vev

  Independent Jev-style decision server with vision input, fine-tuned from Qwen3.5 4B and 9B, that answers Choice, Score, and Noul questions about text, JSON, and images, with screenshots or photos placed directly in state, through a /v1/systemone-compatible server; the repository has Apache-2.0 code and per-set results on public text and image judgment sets, requests are processed on your own GPU after the weights download from Hugging Face, and the weights are CC BY-NC 4.0, for non-commercial use only.

  Read listing →
• 

  VideoJev

  Apache-2.0 open System One video models that locally score temporal candidates and multiple-choice answers over a shared prefix without text generation; video inference requires no hosted API, and the models use their own task interface rather than a TypeSafe-compatible endpoint.

  Read listing →