MixReasoning: Switching Modes to Think

2025-10-07

AI Summary

Standard inference models uniformly apply deep reasoning across all subproblems, resulting in substantial redundancy—only a few critical steps require complex reasoning, while most involve simple operations. To address this, we propose the Dynamic Reasoning Depth (DRD) framework, the first to integrate fine-grained and coarse-grained reasoning paths within a single response. DRD decomposes problems and performs step-level difficulty assessment to adaptively switch between reasoning modes. Crucially, it requires no additional training and relies solely on a lightweight control mechanism. Evaluated on mathematical benchmarks—including GSM8K, MATH-500, and AIME—DRD preserves original accuracy while reducing average reasoning chain length by 37.2%, significantly improving inference efficiency. Its core contribution lies in enabling fine-grained, stepwise control of reasoning depth, thereby departing from conventional fixed-depth inference paradigms.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Reasoning models enhance performance by tackling problems in a step-by-step manner, decomposing them into sub-problems and exploring long chains of thought before producing an answer. However, applying extended reasoning to every step introduces substantial redundancy, as sub-problems vary widely in difficulty and complexity: a small number of pivotal steps are genuinely challenging and decisive for the final answer, while many others only involve straightforward revisions or simple computations. Therefore, a natural idea is to endow reasoning models with the ability to adaptively respond to this variation, rather than treating all steps with the same level of elaboration. To this end, we propose MixReasoning, a framework that dynamically adjusts the depth of reasoning within a single response. The resulting chain of thought then becomes a mixture of detailed reasoning on difficult steps and concise inference on simpler ones. Experiments on GSM8K, MATH-500, and AIME show that MixReasoning shortens reasoning length and substantially improves efficiency without compromising accuracy.

Problem

• Adaptively adjusting reasoning depth for varied sub-problem difficulty
• Reducing redundancy in step-by-step reasoning processes
• Maintaining accuracy while improving computational efficiency

Innovation

• Dynamic reasoning depth adjustment per step
• Mixture of detailed and concise inference modes
• Reduces redundancy while maintaining accuracy