Less is More: Recursive Reasoning with Tiny Networks

2025-10-06

AI Summary

To address the parameter-efficiency and generalization challenges posed by complex reasoning tasks—such as Sudoku, maze solving, and ARC-AGI—for small-scale models, this work proposes the Tiny Recursive Model (TRM): a minimalist recursive neural network with only two layers and 7 million parameters. TRM departs from large-model paradigms by leveraging high-frequency internal recursion to emulate hierarchical reasoning, enabling effective logical inference training on extremely limited data. Its architecture is simpler and more generalizable than that of Hierarchical Recursive Models (HRMs). On the ARC-AGI benchmark, TRM achieves 45% test accuracy on ARC-AGI-1 and 8% on ARC-AGI-2—substantially outperforming state-of-the-art large models like DeepSeek R1 despite using less than 0.01% of their parameters. This work demonstrates the efficacy of lightweight recursive mechanisms for complex reasoning under severe resource constraints, establishing a novel paradigm for efficient AI inference.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Hierarchical Reasoning Model (HRM) is a novel approach using two small neural networks recursing at different frequencies. This biologically inspired method beats Large Language models (LLMs) on hard puzzle tasks such as Sudoku, Maze, and ARC-AGI while trained with small models (27M parameters) on small data (around 1000 examples). HRM holds great promise for solving hard problems with small networks, but it is not yet well understood and may be suboptimal. We propose Tiny Recursive Model (TRM), a much simpler recursive reasoning approach that achieves significantly higher generalization than HRM, while using a single tiny network with only 2 layers. With only 7M parameters, TRM obtains 45% test-accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, higher than most LLMs (e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the parameters.

Problem

• Solving complex puzzles with minimal neural network parameters
• Improving generalization in recursive reasoning models
• Outperforming large language models using tiny architectures

Innovation

• Recursive reasoning with tiny neural networks
• Single tiny network with only two layers
• Outperforms large models using minimal parameters