CRISP: Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning

2026-04-19

AI Summary

This work addresses the high computational cost and latency of long-chain-of-thought reasoning, where existing compression techniques often fail to preserve critical logical steps. The authors propose a fine-grained compression method grounded in the internal attention saliency of the model, which identifies “terminator” tokens as information anchors to precisely prune redundant reasoning steps. Unlike approaches relying on external compressors or coarse-grained strategies, this method operates intrinsically within the model. Evaluated across multiple backbone architectures and mathematical reasoning benchmarks, it achieves token compression rates of 50%–60% while maintaining original reasoning accuracy and preserving both logical coherence and information density.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Data Mining & Knowledge Management: Data Compression
• Natural Language Processing: Summarization

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Long Chain-of-Thought (CoT) reasoning is pivotal for the success of recent reasoning models but suffers from high computational overhead and latency. While prior works attempt to compress CoT via external compressor, they often fail to align with the model's internal reasoning dynamics, resulting in the loss of critical logical steps. This paper presents \textbf{C}ompressing \textbf{R}edundancy in Chain-of-Thought via \textbf{I}ntrinsic \textbf{S}aliency \textbf{P}runing (\textbf{CRISP}), a framework that compresses CoT by exploiting the model's intrinsic saliency. Our analysis reveals a distinct phenomenon: the reasoning termination token \texttt{[object Object]} acts as an information anchor, where its attention pattern effectively demarcates essential reasoning from redundancy. Based on this finding, we design a policy that utilizes these intrinsic attention signals to guide atomic compression operations. In contrast to coarse-grained pruning strategies, CRISP strategically distills the reasoning chain to maximize information density while preserving logical coherence. Empirical results across various backbone models and mathematical datasets demonstrate that CRISP achieves a 50-60% reduction in token count without compromising accuracy, effectively mitigating the efficiency bottleneck of long-context reasoning. We open-source our implementation to facilitate further research in efficient reasoning.

Problem

• Chain-of-Thought
• reasoning compression
• computational overhead
• redundancy
• logical coherence

Innovation

• Chain-of-Thought compression
• intrinsic saliency
• attention-based pruning
• reasoning efficiency
• information density