awesomejev

Community project / Evaluations and independent research

Jev Judge vs Dimension Scores

Independent measurement on three classification tasks: one direct Jev question per row against 12–14 Jev-scored dimensions with locally fitted weights, 5,477 test rows and 34.1M input tokens for $1.43; decomposition reached 0.9076 against 0.8373 on Japanese NLI but flagged about 25× more hard benign rows as attacks, and four repair attempts failed, on dimensions the author wrote himself.

Open the original resource ↗ See it in the full directory Browse all Evaluations and independent research
Awesome Jev share card for Jev Judge vs Dimension Scores Download share card

Share this listing

Maintaining this project? Share its link, badge, or image. Listed means included, not endorsed.

Download share card

This is an independent community listing. Check the source, license, data handling, and evaluation caveats before relying on a project. Inclusion is not an endorsement or security review.

Found an outdated or inaccurate detail? Report a correction →

Explore more in Evaluations and independent research

  • Jev Rerank Bench

    Reranking comparison with raw provider responses, scoring code, dataset-level results, uncertainty intervals, and documented limitations.

    Read listing →

  • Jev Spam Eval

    Exploratory zero-shot spam study against trained TF-IDF baselines, including results and explicit post-hoc-tuning caveats.

    Read listing →

  • jev-calibration-audit

    Independent Jev-1.13.0 audit with reproducible code and per-call JSONL: tests abstention options, matched Korean and English items, question-shape interference, and option order; its strong abstention finding is on the KoBBQ dataset, not a universal calibration guarantee.

    Read listing →