🤖 AI Summary
This work addresses the performance degradation and increased computational overhead in small language models (SLMs) caused by “overthinking” during chain-of-thought (CoT) distillation from large language models. To mitigate reasoning drift, the authors propose a structured reasoning framework that introduces control tokens—such as <TEMP_LOW> and <TEMP_HIGH>—to guide SLMs along efficient, ordered reasoning pathways. The approach integrates control-token-driven CoT distillation, structured trajectory optimization, and few-shot fine-tuning. Evaluated on Qwen3-8B with only 5,000 training samples, the method achieves a 9.9% absolute accuracy gain on GPQA-diamond and a 9.1% improvement on zero-shot MMLU-Pro, while substantially reducing token consumption. These results demonstrate the model’s enhanced capacity to internalize structured reasoning patterns.
📝 Abstract
Chain-of-Thought (CoT) distillation from Large Language Models (LLMs) often induces "overthinking" in Small Language Models (SLMs), leading to performance degradation and excessive token consumption. In this study, we propose Disciplined Chain-of-Thought (D-CoT), a novel framework that enforces a structured reasoning process using control tags -- such as <TEMP_LOW> for fact-checking and <TEMP_HIGH> for multi-perspective exploration -- as auxiliary scaffolding during training. By optimizing the CoT trajectory, D-CoT suppresses reasoning drift and simultaneously achieves token reduction and performance improvement. We demonstrate the efficacy of our approach on Qwen3-8B: with only 5,000 training samples, D-CoT significantly boosts accuracy on GPQA-diamond by 9.9% and MMLU-Pro (0-shot) by 9.1%, while drastically reducing computational costs. Furthermore, we confirm that the model internalizes this disciplined thought structure, maintaining high performance even without explicit control tags during inference.