From LLMs to LRMs: Rethinking Pruning for Reasoning-Centric Models
This work addresses the limitations of existing model pruning methods, which are primarily designed for instruction-following large language models and struggle to effectively adapt to reasoning-augmented models that explicitly generate long reasoning chains. The study systematically investigates pruning strategies tailored for both model types, proposing a calibration and recovery pipeline aligned with the original training distribution. It evaluates static depth pruning, static width pruning, and dynamic pruning across 17 tasks, revealing—for the first time—the dependence of pruning efficacy on the underlying reasoning paradigm. The findings indicate that depth pruning is better suited for classification tasks, while width pruning demonstrates greater robustness in generative and reasoning tasks. Moreover, static pruning better preserves reasoning capabilities, whereas dynamic pruning remains challenging for long-chain reasoning scenarios.