When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning

2025-05-21

AI Summary

Large reasoning models (LRMs) suffer from “overthinking,” degrading inference efficiency. This work addresses the problem via reinforcement learning (RL) training coupled with multi-dimensional behavioral analysis—including attention visualization, termination confidence modeling, clustering, and causal attribution—to systematically characterize LRM reasoning behaviors under “think less” prompting. We identify, for the first time, three distinct reasoning modalities: no-thinking, explicit thinking, and implicit thinking. Results show that implicit thinking achieves high accuracy while substantially compressing response length; no-thinking shortens outputs but severely harms accuracy; and a fundamental trade-off exists among accuracy, response length, and termination confidence across modalities. Crucially, we uncover modality-specific divergence in the RL-trained thinking-termination mechanism—revealing intrinsic inconsistencies in current optimization objectives. This work establishes a novel paradigm for efficient, trustworthy, and interpretable reasoning control in LRMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Large reasoning models (LRMs) have significantly advanced performance on complex tasks, yet their tendency to overthink introduces inefficiencies. This study investigates the internal mechanisms of reinforcement learning (RL)-trained LRMs when prompted to save thinking, revealing three distinct thinking modes: no thinking (NT), explicit thinking (ET), and implicit thinking (IT). Through comprehensive analysis of confidence in thinking termination, attention from thinking to generation, and attentional focus on input sections, we uncover key factors influencing the reasoning behaviors. We further find that NT reduces output length at the cost of accuracy, while ET and IT maintain accuracy with reduced response length. Our findings expose fundamental inconsistencies in RL-optimized LRMs, necessitating adaptive improvements for reliable efficiency.

Problem

• Investigates inefficiencies in large reasoning models due to overthinking
• Analyzes three thinking modes: no, explicit, and implicit thinking
• Examines accuracy and output length trade-offs in reasoning behaviors

Innovation

• Reinforcement learning-trained large reasoning models
• Three thinking modes: NT, ET, IT
• Analyzes confidence, attention, and focus factors