TwT: Thinking without Tokens by Habitual Reasoning Distillation with Multi-Teachers' Guidance

2025-03-31

AI Summary

To address the excessive output token count and high inference overhead in large language models (LLMs) caused by explicit chain-of-thought reasoning, this paper proposes Habitual Reasoning Distillation (HRD). HRD leverages multi-teacher collaborative supervision and Dual-Criterion Rejection Sampling (DCRS) to construct high-quality, unsupervised distillation data, internalizing complex reasoning into intuitive, habitual model behavior—enabling a “token-free thinking” lightweight inference paradigm. Crucially, HRD eliminates intermediate reasoning tokens during inference while preserving or even improving task accuracy (up to +13.6%). It significantly reduces output token count, inference latency, and computational cost, making it suitable for edge devices and high-concurrency deployments. The core contribution lies in pioneering the integration of cognitively inspired habit formation mechanisms into knowledge distillation, establishing an efficient, implicit-reasoning inference paradigm without explicit reasoning traces.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Natural Language Processing: (Large) Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large Language Models (LLMs) have made significant strides in problem-solving by incorporating reasoning processes. However, this enhanced reasoning capability results in an increased number of output tokens during inference, leading to higher computational costs. To address this challenge, we propose TwT (Thinking without Tokens), a method that reduces inference-time costs through habitual reasoning distillation with multi-teachers' guidance, while maintaining high performance. Our approach introduces a Habitual Reasoning Distillation method, which internalizes explicit reasoning into the model's habitual behavior through a Teacher-Guided compression strategy inspired by human cognition. Additionally, we propose Dual-Criteria Rejection Sampling (DCRS), a technique that generates a high-quality and diverse distillation dataset using multiple teacher models, making our method suitable for unsupervised scenarios. Experimental results demonstrate that TwT effectively reduces inference costs while preserving superior performance, achieving up to a 13.6% improvement in accuracy with fewer output tokens compared to other distillation methods, offering a highly practical solution for efficient LLM deployment.

Problem

• Reducing LLM inference costs by minimizing output tokens
• Maintaining high performance through habitual reasoning distillation
• Enhancing accuracy with fewer tokens via multi-teacher guidance

Innovation

• Habitual Reasoning Distillation method internalizes reasoning
• Dual-Criteria Rejection Sampling ensures dataset quality
• Multi-teacher guidance reduces inference costs effectively