Recursive Models for Long-Horizon Reasoning

2026-03-02

AI Summary

This work addresses the challenge that large language models struggle with long-horizon reasoning due to limited context windows. To overcome this, the authors propose a recursive self-invocation architecture that leverages a context isolation mechanism to recursively decompose complex tasks into subproblems, achieving exponentially effective context compression. The approach formalizes recursion as a general framework for solving computable problems and theoretically demonstrates its optimal representational capacity within generalized agent systems. Empirical evaluations show that the proposed 3B-parameter model significantly outperforms state-of-the-art large language models on long-horizon combinatorial search tasks, such as Boolean satisfiability.

Technology Category

• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Modern language models reason within bounded context, an inherent constraint that poses a fundamental barrier to long-horizon reasoning. We identify recursion as a core principle for overcoming this barrier, and propose recursive models as a minimal realization, where the model can recursively invoke itself to solve subtasks in isolated contexts. We prove that any computable problem admits a recursive decomposition in which each subtask requires only exponentially smaller active context than standard autoregressive models; this strictly surpasses any context management approach confined to a single sequence, such as summarization. We further generalize our framework to modern agentic systems with arbitrary context processing and control flows, and prove that recursive models can achieve optimal power within this broader class. Experimentally, we train a 3B model to reason recursively and evaluate on Boolean satisfiability, a task requiring long-horizon combinatorial search, where it significantly outperforms frontier LLMs.

Problem

• long-horizon reasoning
• bounded context
• recursive decomposition
• combinatorial search
• language models

Innovation

• recursive models
• long-horizon reasoning
• context compression
• combinatorial search
• agentic systems