A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning

2025-10-17

AI Summary

This work addresses the lack of theoretical foundations for test-time scaling methods in large language model (LLM) inference. We propose the first theoretical analysis framework for LLM reasoning, grounded in confidence estimation, which formally characterizes the error sources of self-consistency and perplexity-based approaches. Building on this analysis, we introduce Perplexity-Consistency (PC), a novel mechanism that synergistically integrates the strengths of both paradigms, augmented with inference-path pruning to suppress low-probability distracting tokens. We provide rigorous theoretical guarantees showing that PC achieves exponential improvement in error convergence rate. Extensive evaluation across seven benchmarks demonstrates that PC matches self-consistency in reasoning accuracy, achieves superior confidence calibration, reduces sampling cost by 50%, and significantly enhances both inference efficiency and reliability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Test-time scaling seeks to improve the reasoning performance of large language models (LLMs) by adding computational resources. A prevalent approach within the field is sampling-based test-time scaling methods, which enhance reasoning by generating multiple reasoning paths for a given input during inference. However, despite its practical success, the theoretical foundations remain underexplored. In this paper, we provide the first theoretical framework for analyzing sampling-based test-time scaling methods, grounded in the perspective of confidence estimation. Based on the framework, we analyze two dominant paradigms: self-consistency and perplexity, and reveal key limitations: self-consistency suffers from high estimation error while perplexity exhibits substantial modeling error and possible degradation of the estimation error convergence. To address these limitations, we introduce RPC, a hybrid method that leverages our theoretical insights through two key components: Perplexity Consistency and Reasoning Pruning. Perplexity Consistency combines the strengths of self-consistency and perplexity, boosting the convergence rate of estimation error from linear to exponential while preserving model error. Reasoning Pruning prevents degradation by eliminating low-probability reasoning paths. Both theoretical analysis and empirical results across seven benchmark datasets demonstrate that RPC has a strong potential for reducing reasoning error. Notably, RPC achieves reasoning performance comparable to self-consistency while not only enhancing confidence reliability but also reducing sampling costs by 50%. The code and resources are available at https://wnjxyk.github.io/RPC.

Problem

• Developing theoretical framework for sampling-based test-time scaling methods
• Addressing limitations of self-consistency and perplexity in LLM reasoning
• Proposing hybrid method to reduce estimation error and sampling costs

Innovation

• Hybrid method combining perplexity and self-consistency
• Prunes low-probability reasoning paths to prevent degradation
• Boosts estimation error convergence rate exponentially