🤖 AI Summary
Large language models (LLMs) remain constrained in complex logical and mathematical reasoning. This paper introduces Cumulative Reasoning (CR), a novel framework that emulates human incremental thinking by decomposing problems into subtasks and iteratively accumulating and reusing intermediate propositions and execution results—without external retrieval or internet access. CR integrates symbolic reasoning chain construction, iterative prompt engineering, and an internal code execution environment to enable pure model-intrinsic reasoning leaps. Experiments demonstrate substantial gains: 98.04% accuracy on FOLIO (+9.3%), 98% on Game of 24 (+24%), +4.2% overall on MATH and up to +43% on its hardest subset. A code-only CR variant achieves 72.2%, significantly outperforming PAL and PoT (+38.8%). This work is the first to systematically propose and empirically validate an intrinsic, cumulative, iterative reasoning paradigm.
📝 Abstract
Despite the recent advancements in language models (LMs), their ability to solve complex problems remains limited. This paper introduces Cumulative Reasoning (CR), a novel approach that utilizes LMs cumulatively and iteratively, mirroring human thought processes for problem-solving. CR decomposes tasks into smaller, manageable components and leverages previous propositions for effective composition, significantly enhancing problem-solving capabilities. We demonstrate CR's superiority through several complex reasoning tasks: it outperforms existing methods in logical inference tasks with up to a 9.3% improvement, achieving 98.04% accuracy on the curated FOLIO wiki dataset. In the Game of 24, it achieves 98% accuracy, marking a 24% improvement over the prior state-of-the-art. Additionally, CR sets new state-of-the-art on the MATH dataset, achieving a 4.2% increase from previous methods and a 43% relative improvement in the most challenging problems. By extending CR to incorporate a code environment without external aids like retrieval or web browsing, we further harness the computational and logical reasoning capabilities of LMs, achieving a remarkable 72.2% accuracy on the MATH dataset and outperforming the PAL/PoT method by 38.8%. Our work not only sets new state-of-the-art but also paves the way toward more sophisticated AI reasoning methods. The code is available at https://github.com/iiis-ai/cumulative-reasoning.
