Towards Understanding Multi-Round Large Language Model Reasoning: Approximability, Learnability and Generalizability

2025-03-05

AI Summary

This paper establishes a theoretical foundation for multi-step reasoning in large language models (LLMs), addressing the core question of how iterative autoregressive inference enhances approximation capability, learnability, and generalization. Methodologically, it extends the PAC learning framework to sequence-to-sequence generation for the first time; proves that finite-context Transformers, via multi-step reasoning, can universally approximate any Turing-computable sequence-to-sequence function; and rigorously quantifies error propagation across reasoning steps. The main contributions are threefold: (1) establishing universal approximation guarantees for multi-step reasoning; (2) ensuring learnability under bounded context constraints; and (3) providing generalization bounds with controllable error accumulation. Collectively, these results deliver the first rigorous, unified theoretical justification for cognitively inspired, multi-step reasoning in LLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Recent advancements in cognitive science and multi-round reasoning techniques for Large Language Models (LLMs) suggest that iterative thinking processes improve problem-solving performance in complex tasks. Inspired by this, approaches like Chain-of-Thought, debating, and self-refinement have been applied to auto-regressive LLMs, achieving significant successes in tasks such as mathematical reasoning, commonsense reasoning, and multi-hop question answering. Despite these successes, the theoretical basis for how multi-round reasoning enhances problem-solving abilities remains underexplored. In this work, we investigate the approximation, learnability, and generalization properties of multi-round auto-regressive models. We show that Transformers with finite context windows are universal approximators for steps of Turing-computable functions and can approximate any Turing-computable sequence-to-sequence function through multi-round reasoning. We extend PAC learning to sequence generation and demonstrate that multi-round generation is learnable even when the sequence length exceeds the model's context window. Finally, we examine how generalization error propagates across rounds, and show how the aforementioned approaches can help constrain this error, ensuring outputs stay within an expectation boundary. This work sheds light on the systemic theoretical foundations of multi-round sequence learning and reasoning, emphasizing its role in inference complexity.

Problem

• Explores theoretical basis of multi-round reasoning in LLMs.
• Investigates approximation, learnability, and generalization in auto-regressive models.
• Analyzes error propagation and constraints in multi-round reasoning.

Innovation

• Transformers approximate Turing-computable functions iteratively.
• PAC learning extended to multi-round sequence generation.
• Generalization error constrained via multi-round reasoning techniques.