Valid $\ne$ Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought

2026-07-13

AI Summary

This work addresses the inefficiency in chain-of-thought reasoning by large language models, where logically correct but redundant or irrelevant steps lead to unnecessary computational overhead—a problem overlooked by existing evaluation metrics that fail to distinguish “valid yet non-essential” reasoning. The study formally defines and systematically diagnoses this issue, introducing RIV-GSM8K, a diagnostic benchmark encompassing five categories of inefficient reasoning patterns. It further proposes CAID, a training-free information-theoretic metric that quantifies the information density of reasoning steps, and integrates it with PACE, a post-hoc compression strategy. Evaluated on GSM8K, StrategyQA, and ARC-Challenge, this approach achieves 31%–53% token savings while preserving accuracy, significantly outperforming baseline methods such as random pruning and PRM.

Abstract

Chain-of-Thought (CoT) prompting has significantly advanced the reasoning capabilities of Large Language Models (LLMs), yet it often incurs substantial computational costs due to over-reasoning: the generation of redundant, verbose, or irrelevant steps. While existing reasoning step evaluators effectively detect logical fallacies and factual errors, our analysis reveals a critical blind spot: they fail to penalize valid but inefficient reasoning steps that inflate token usage without contributing to the solution. To systematically diagnose this limitation, we introduce RIV-GSM8K, a diagnostic benchmark injected with five distinct types of inefficiencies, including circular reasoning and excessive decomposition. Diagnostic experiments reveal that state-of-the-art evaluators struggle to distinguish these inefficiencies from necessary reasoning. To address this gap, we propose CAID (Context-Aware Information Density), a training-free metric grounded in information theory that identifies low-utility steps. To validate the metric's practical utility, we apply it within PACE, a post-hoc compression strategy. Additional control experiments show that the gains of PACE are not explained by trivial pruning: compared with random step removal and PRM-based compression baselines, it preserves accuracy at substantially higher compression rates. Empirical results on GSM8K, StrategyQA, and ARC-Challenge demonstrate that PACE reduces token consumption by 31-53% while maintaining accuracy, confirming that CAID successfully distills informational froth from reasoning chains without compromising deductive validity.

Problem

• Chain-of-Thought
• reasoning efficiency
• token consumption
• inefficiency diagnosis
• large language models

Innovation

• Chain-of-Thought
• Information Density
• Reasoning Efficiency
• Token Compression
• Diagnostic Benchmark