Recursive Agent Optimization

2026-05-07

AI Summary

Current large language models are constrained by fixed context windows, limiting their ability to handle highly complex tasks. This work proposes a reinforcement learning–based recursive agent training framework that enables agents, during inference, to autonomously decide whether and how to recursively invoke themselves, dynamically decomposing tasks and delegating subtasks. The approach achieves, for the first time, adaptive recursion and coordination among agents at inference time, effectively circumventing context length limitations. It significantly enhances generalization and reasoning efficiency on tasks far exceeding the complexity encountered during training, while maintaining higher training efficiency and achieving lower overall inference latency compared to single-agent systems.

Abstract

We introduce Recursive Agent Optimization (RAO), a reinforcement learning approach for training recursive agents: agents that can spawn and delegate sub-tasks to new instantiations of themselves recursively. Recursive agents implement an inference-time scaling algorithm that naturally allows agents to scale to longer contexts and generalize to more difficult problems via divide-and-conquer. RAO provides a method to train models to best take advantage of such recursive inference, teaching agents when and how to delegate and communicate. We find that recursive agents trained in this way enjoy better training efficiency, can scale to tasks that go beyond the model's context window, generalize to tasks much harder than the ones the agent was trained on, and can enjoy reduced wall-clock time compared to single-agent systems.

Problem

• Recursive Agents
• Reinforcement Learning
• Inference-time Scaling
• Task Decomposition
• Context Window Extension

Innovation

• Recursive Agent Optimization
• reinforcement learning
• recursive agents
• divide-and-conquer
• inference-time scaling