PAC Reasoning: Controlling the Performance Loss for Efficient Reasoning

2025-10-10

AI Summary

Large reasoning models (LRMs) face dual challenges in deployment: prohibitively high computational overhead and uncontrolled performance degradation. To address these, we propose a PAC-inspired dynamic mode-switching inference framework. Our method constructs a distribution-agnostic, uncertainty-driven confidence upper bound—yielding, for the first time, a statistically rigorous performance-loss upper bound for adaptive switching between “thinking” and “non-thinking” inference modes. We further design a context-aware thresholding mechanism that dynamically selects optimal inference paths based on input characteristics. Evaluated across multiple benchmarks, our approach reduces computational cost by up to 47% while strictly guaranteeing that performance loss remains within user-specified tolerance thresholds. Theoretical analysis ensures PAC-style generalization guarantees, and empirical results validate both practical efficacy and implementation robustness. This work bridges statistical learning theory with efficient LRM inference, offering a principled, deployable solution to the compute–accuracy trade-off.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Large reasoning models (LRMs) have achieved remarkable progress in complex problem-solving tasks. Despite this success, LRMs typically suffer from high computational costs during deployment, highlighting a need for efficient inference. A popular direction of efficiency improvement is to switch the LRM between thinking and nonthinking modes dynamically. However, such approaches often introduce additional reasoning errors and lack statistical guarantees for the performance loss, which are critical for high-stakes applications. In this work, we propose Probably Approximately Correct (PAC) reasoning that controls the performance loss under the user-specified performance loss tolerance. In particular, we construct an upper confidence bound on the performance loss, formulated as a monotone function of the uncertainty score, and subsequently determine a threshold for switching to the nonthinking model. Theoretically, using the threshold to switch between the thinking and nonthinking modes ensures bounded performance loss in a distribution-free manner. Our comprehensive experiments on reasoning benchmarks show that the proposed method can save computational budgets and control the user-specified performance loss.

Problem

• Control performance loss in large reasoning models
• Ensure statistical guarantees for efficient inference
• Dynamically switch between thinking and non-thinking modes

Innovation

• PAC reasoning controls performance loss tolerance
• Threshold switching based on uncertainty score
• Distribution-free bounded performance loss guarantee