Community project / Evaluations and independent research

Love-Language Arena

Experimental local reproduction of the Choice, Score, and Noul pattern on Ollama logprobs, with order-reversed and negated re-asks to expose position bias, used to test three open models as fully crossed judges of Chinese and English rewrites, with self-preference correction, per-judge Platt calibration, dev/holdout prompt selection, and raw per-call JSONL. It does not call Jev, and its validation labels are Claude-generated rather than human, so the judge pass rates and model ranking show that the pipeline runs, not that the judges are valid.

Open the original resource ↗ See it in the full directory Start a reading list → Browse all Evaluations and independent research

Download share card ↓

This is an independent community listing. Check the source, license, data handling, and evaluation caveats before relying on a project. Inclusion is not an endorsement or security review.

Found an outdated or inaccurate detail? Report a correction →

Explore more in Evaluations and independent research

• 

  Luce

  Open recipe for Jev-style decision models: a task description, an LLM teacher that writes the data, then LoRA plus a decision head on Qwen3-4B-Base returning calibrated choice, score, and boolean probabilities on a 12 GB GPU; the README reports accuracy and ECE against Jev on identical test items, including where training does not help, with a GPU-free replay demo.

  Read listing →
• 

  NanoJev

  Open 0.6B Jev-style model for one-pass action probabilities in Maze, Snake, and ViZDoom, with public checkpoint, training data, and replayable game comparisons; it is specialized to those games and is not a general Jev API replacement.

  Read listing →
• 

  Open Alternative to Jev

  Apache-licensed Python library that reads option-token probabilities from open models through Transformers or vLLM, with packed and separate question modes, temperature scaling, benchmark scripts, and raw result files; its measurements compare modes of the same open model rather than Jev, packed answers can change with question order, and the package is not a drop-in System One API server.

  Read listing →