TL;DR: Too Long, Do Re-weighting for Effcient LLM Reasoning Compression

2025-06-03

AI Summary

To address the efficiency bottleneck posed by lengthy output generation in large language models (LLMs) during complex reasoning, this paper proposes a dynamic ratio reweighting training paradigm grounded in dual-process cognitive theory. The method adaptively prunes reasoning chains by online adjusting training weights between System-1 (intuitive) and System-2 (deliberative) data—requiring no human annotations, auxiliary models, or ensemble techniques. Its core innovations are (i) a learnable dynamic weight scheduling mechanism and (ii) a synergistic Chain-of-Thought distillation framework for joint optimization. Evaluated on the DeepSeek-R1-Distill model series, the approach achieves an average 39.7% reduction in output token count while preserving inference accuracy—yielding substantial gains in throughput and deployment feasibility for long-reasoning tasks.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models (LLMs) have recently achieved remarkable progress by leveraging Reinforcement Learning and extended Chain-of-Thought (CoT) techniques. However, the challenge of performing efficient language reasoning--especially during inference with extremely long outputs--has drawn increasing attention from the research community. In this work, we propose a dynamic ratio-based training pipeline that does not rely on sophisticated data annotations or interpolation between multiple models. We continuously balance the weights between the model's System-1 and System-2 data to eliminate redundant reasoning processes while preserving the model's reasoning capability. We validate our approach across models on DeepSeek-R1-Distill-7B and DeepSeek-R1-Distill-14B and on a diverse set of benchmarks with varying difficulty levels. Our method significantly reduces the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. Our code and data will be available soon.

Problem

• Efficient language reasoning with long outputs
• Reducing redundant reasoning processes in LLMs
• Maintaining accuracy while compressing output tokens

Innovation

• Dynamic ratio-based training pipeline
• Balancing System-1 and System-2 weights
• Reduces output tokens by 40%