PreSTO: Predictive Subtree Prefetching for Fast LLM Power Sampling

Key points

• Its sampler, PowerMH, is slow because its Metropolis--Hastings (MH) updates are sequential: each accept/reject decision sets the state from which the next proposal is generated.1
• We propose Predictive SubTree Prefetching (PreSTO), which accelerates PowerMH by prefetching future proposals and provably leaves its MH transition kernel unchanged.1
• Across seven datasets and five base LLMs, PreSTO achieves median speedups of $1.71\times$, $1.91\times$, and $2.33\times$ on PowerMH and its follow-ups EntropyCut and MultiTryMH, respectively; on the first two, it makes a median of $3.9\times$ fewer model calls, each only $2.1\times$ longer.1
• On LCB V6 with Qwen3.5-9B, the predictive traversal rule is a further $1.25\times$--$1.37\times$ faster than the default breadth-first traversal.1

Sources (2)

• [1]PreSTO: Predictive Subtree Prefetching for Fast LLM Power Sampling
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 07:58 AM

  Power sampling improves large language model (LLM) reasoning at test time by sampling from a sharpened version of the base model's distribution.

  Its sampler, PowerMH, is slow because its Metropolis--Hastings (MH) updates are sequential: each accept/reject decision sets the state from which the next proposal is generated.

• [2]PreSTO: Predictive Subtree Prefetching for Fast LLM Power Sampling
  Hugging Face Daily Papers · Oct 9, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.