Readout Stability in Prefill-Only Decision Models:Zero-Label Prediction and Inference-Time Compute Allocation

Key points

• We show that this read-out structure comes with a testable property.1
• When an intervention changes only the candidate menu and leaves the input text fixed, the post-intervention accuracy is already determined by the cached first-pass distribution.1
• A probability-level variant of the same estimator errs by 21.0 points, so the property lives in the ranking rather than in the probabilities and is not recovered by calibration.1
• Same-scale generative language models do not share the property.1

Sources (1)

• [1]Readout Stability in Prefill-Only Decision Models:Zero-Label Prediction and Inference-Time Compute Allocation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 04:10 AM

  Prefill-only decision models inspired by the Jev model score every candidate in a menu during a single forward pass and never decode, which makes one call one to two orders of magnitude cheaper than a same-scale generative language model.

  We show that this read-out structure comes with a testable property.

Extractive summary: sentences quoted from the sources.