Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models

2026-06-12

AI Summary

This work addresses the tendency of large reasoning models to generate redundant outputs and suffer accuracy degradation during inference due to overthinking. To mitigate this issue, the authors propose ASAG, a training-free, adaptive generation method that dynamically monitors the model’s reasoning state through its attention distribution. Leveraging this signal, ASAG implements a plug-and-play adaptive early-stopping mechanism without requiring additional training or handcrafted prompts. Extensive experiments across nine benchmarks demonstrate that ASAG consistently enhances the performance of prominent large reasoning models: for instance, Qwen3-8B achieves an average accuracy gain of 3.2% while reducing generated tokens by nearly 40%.

Abstract

By incorporating test-time compute scaling, large reasoning models (LRMs) can solve complex problems through explicit chain-of-thought (CoT) reasoning processes. However, they often suffer from overthinking, resulting in redundant token outputs and degraded accuracy. Current methods to mitigate this issue remain limited: training-based approaches require substantial computational resources, while training-free methods rely on well-crafted prompts or unreliable confidence signals. In this work, we investigate early stopping from the perspective of attention distributions and propose a simple method, ASAG, which infers the model's reasoning state and adaptively adjusts the generation strategy. The proposed framework is training-free and plug-and-play, enabling seamless integration into existing LRMs. Extensive experiments on nine benchmarks demonstrate consistent improvements across mainstream LRMs with varying parameter scales, including the DeepSeek-R1-Distill and Qwen3 series. Specifically, ASAG improves average accuracy by 3.2% while reducing the number of generated tokens by nearly 40% across all reasoning tasks on Qwen3-8B.

Problem

• overthinking
• reasoning models
• early stopping
• chain-of-thought
• token redundancy

Innovation

• early stopping
• attention distribution
• chain-of-thought reasoning
• training-free method
• adaptive generation