From Probabilities to Decisions: Search and Multi-Teacher Distillation with Jev

Key points

• Probability-only models, which TypeSafe calls System One models, return calibrated probabilities for fixed choices in milliseconds and generate no text.1
• We study one such model, Jev, through two tasks that require decisions under tight constraints.1
• Live model calls are too slow for search, so we distill pairwise judgments into a compact evaluator that runs at every position.1
• In chess, averaging both judges' labels beats spending the whole budget on Qwen alone by 9.6 Elo (95% interval 4.3 to 14.9), and the gain replicates on fresh openings; a second answer from the same judge is no substitute, and Jev is the strongest partner for Qwen among the models tested.1

Sources (1)

• [1]From Probabilities to Decisions: Search and Multi-Teacher Distillation with Jev
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 10:37 PM

  In bullet chess, a bot that places Jev's judgment inside Stockfish search alongside an opening book and endgame tablebases climbs above a 2200 Lichess bullet rating against other bots.

  Probability-only models, which TypeSafe calls System One models, return calibrated probabilities for fixed choices in milliseconds and generate no text.

Extractive summary: sentences quoted from the sources.