Recursive Multi-Agent Systems

2026-04-28

AI Summary

This work proposes RecursiveMAS, a novel framework that extends recursive computation from single-agent to multi-agent systems to enhance collaborative reasoning efficiency in complex tasks. By introducing a lightweight RecursiveLink module, the framework establishes a unified latent-space recursion loop that enables cross-agent hidden state propagation and joint inference. Training stability and efficiency are ensured through a joint inner-outer loop optimization algorithm and a shared-gradient-based credit assignment mechanism. Experimental results demonstrate that RecursiveMAS achieves an average accuracy improvement of 8.3% across nine benchmarks spanning mathematics, science, medicine, search, and code generation, while accelerating end-to-end inference by 1.2–2.4× and reducing token consumption by 34.6%–75.6%.

Abstract

Recursive or looped language models have recently emerged as a new scaling axis by iteratively refining the same model computation over latent states to deepen reasoning. We extend such scaling principle from a single model to multi-agent systems, and ask: Can agent collaboration itself be scaled through recursion? To this end, we introduce RecursiveMAS, a recursive multi-agent framework that casts the entire system as a unified latent-space recursive computation. RecursiveMAS connects heterogeneous agents as a collaboration loop through the lightweight RecursiveLink module, enabling in-distribution latent thoughts generation and cross-agent latent state transfer. To optimize our framework, we develop an inner-outer loop learning algorithm for iterative whole-system co-optimization through shared gradient-based credit assignment across recursion rounds. Theoretical analyses of runtime complexity and learning dynamics establish that RecursiveMAS is more efficient than standard text-based MAS and maintains stable gradients during recursive training. Empirically, we instantiate RecursiveMAS under 4 representative agent collaboration patterns and evaluate across 9 benchmarks spanning mathematics, science, medicine, search, and code generation. In comparison with advanced single/multi-agent and recursive computation baselines, RecursiveMAS consistently delivers an average accuracy improvement of 8.3%, together with 1.2$\times$-2.4$\times$ end-to-end inference speedup, and 34.6%-75.6% token usage reduction. Code and Data are provided in https://recursivemas.github.io.

Problem

• Recursive Multi-Agent Systems
• Agent Collaboration
• Latent-Space Recursion
• Scalable Reasoning
• Multi-Agent Coordination

Innovation

• Recursive Multi-Agent Systems
• Latent-Space Recursion
• RecursiveLink
• Gradient-Based Credit Assignment
• Collaborative Reasoning