Constructing Pareto Frontiers of Optimal Reasoning Termination Methods for Overthinking in Reasoning Models

Key points

• Reasoning models can benefit from test-time scaling, but continued reasoning may become ineffective or harmful, a phenomenon termed overthinking.1
• Evaluations against full thinking compare accuracy and token usage but lack references for overthinking severity and strategies' theoretical mitigation potential.1
• Using each trajectory's earliest measured position attaining maximum empirical accuracy as a reference, we define Overthinking Ratio and Accuracy Degradation as the fraction of thinking tokens generated afterward and the accuracy decrease from this reference to full thinking, respectively.1
• Based on this unified framework, we use the complete STP measurements to construct a token-accuracy Pareto frontier for each strategy under a common normalized thinking-token budget.1

Sources (1)

• [1]Constructing Pareto Frontiers of Optimal Reasoning Termination Methods for Overthinking in Reasoning Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 01:22 PM

  To characterize overthinking at the model-task level, we first introduce Step-wise Truncation Probing (STP), repeatedly generating answers from multiple retained prefixes along each fixed sampled reasoning trajectory.

  Reasoning models can benefit from test-time scaling, but continued reasoning may become ineffective or harmful, a phenomenon termed overthinking.

Extractive summary: sentences quoted from the sources.