Verbosity-Aware Rationale Reduction: Effective Reduction of Redundant Rationale via Principled Criteria

2024-12-30

AI Summary

To address excessive computational overhead in large language model (LLM) inference caused by redundant intermediate reasoning steps, this paper proposes a sentence-level rationale compression framework. Unlike prevailing token-level compression methods, our approach pioneers sentence-level granularity, jointly modeling sentence likelihood and redundancy to construct an interpretable, learnable verbosity scoring mechanism, integrated with supervised distillation-based pruning for efficient inference. Experiments across multiple models (e.g., LLaMA-2, Qwen) and reasoning-intensive benchmarks (e.g., GSM8K, HotpotQA) demonstrate an average 17.15% reduction in generated sequence length, substantial decreases in inference latency and GPU memory consumption, and zero accuracy degradation. Our key contributions are: (1) the first sentence-level rationale compression paradigm; (2) a likelihood-based, interpretable redundancy metric; and (3) an end-to-end compression framework that simultaneously preserves answer accuracy and enhances inference efficiency.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Large Language Models (LLMs) rely on generating extensive intermediate reasoning units (e.g., tokens, sentences) to enhance final answer quality across a wide range of complex tasks. While generating multiple reasoning paths or iteratively refining rationales proves effective for improving performance, these approaches inevitably result in significantly higher inference costs. In this work, we propose a novel sentence-level rationale reduction training framework that leverages likelihood-based criteria, verbosity, to identify and remove redundant reasoning sentences. Unlike previous approaches that utilize token-level reduction, our sentence-level reduction framework maintains model performance while reducing generation length. This preserves the original reasoning abilities of LLMs and achieves an average 17.15% reduction in generation costs across various models and tasks.

Problem

• Large Language Models
• Computation Cost
• Time Efficiency

Innovation

• Sentence-level Optimization
• Cost Reduction
• Adaptive Model Efficiency