Thinking Inertia: LLMs Keep Thinking When Told Not To

Key points

• Large Language Models (LLMs) increasingly ship with explicit "thinking modes", yet their counterpart, "no-thinking", has received far less attention.1
• We instead normalize each response into a pre-answer trace and final answer, and evaluate it at three levels: (i) Empty-Thinking Rate for strict answer-only compliance; (ii) instruction-aware Question-Pre-answer Relevance for similarity between the question and pre-answer trace; and (iii) LLM-as-judge Explicit Inference Rate for visible explicit inference.1
• We evaluate six prompting interventions on six LLMs across Boolean, multiple-choice, and open-ended questions.1
• We find that explicit no-think controls cannot reliably eliminate visible inference.1

Sources (1)

• [1]Thinking Inertia: LLMs Keep Thinking When Told Not To
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 11:49 AM

  We study LLMs' no-thinking behavior along two axes. a.

  Large Language Models (LLMs) increasingly ship with explicit "thinking modes", yet their counterpart, "no-thinking", has received far less attention.

Extractive summary: sentences quoted from the sources.