Efficient Agentic Reasoning Through Self-Regulated Simulative Planning

2026-05-21

AI Summary

Current intelligent agents often suffer from computational inefficiency and unstable accuracy gains due to inadequate control over the timing, structure, and scope of planning. To address this, this work proposes the SR²AM framework, which decouples decision-making into three distinct modules: simulated reasoning, self-regulation, and reactive execution—explicitly separating planning from execution within large language models for the first time. A learnable self-regulation mechanism adaptively invokes a unified planning module as needed, leveraging a world model to predict future states. Experiments demonstrate that SR²AM-v1.0-30B matches the performance of models with 685B–1T parameters across multiple tasks while reducing inference tokens by 25.8%–95.3%. Furthermore, reinforcement learning fine-tuning increases the average planning horizon by 22.8% with only a 2.0% rise in planning frequency.

Abstract

How should an agent decide when and how to plan? A dominant approach builds agents as reactive policies with adaptive computation (e.g., chain-of-thought), trained end-to-end expecting planning to emerge implicitly. Without control over the presence, structure, or horizon of planning, these systems dramatically increase reasoning length, yielding inefficient token use without reliable accuracy gains. We argue efficient agentic reasoning benefits from decomposing decision-making into three systems: simulative reasoning (System II) grounding deliberation in future-state prediction via a world model; self-regulation (System III) deciding when and how deeply to plan via a learned configurator; and reactive execution (System I) handling fine-grained action. Simulative reasoning provides unified planning across diverse tasks without per-domain engineering, while self-regulation ensures the planner is invoked only when needed. To test this, we develop SR$^2$AM (Self-Regulated Simulative Reasoning Agentic LLM), realizing both as distinct stages within an LLM's chain-of-thought, with the LLM as world model. We explore two instantiations: recording decisions from a prompted multi-module system (v0.1) and reconstructing structured plans from traces of pretrained reasoning LLMs (v1.0), trained via supervised then reinforcement learning (RL). Across math, science, tabular analysis, and web information seeking, v0.1-8B and v1.0-30B achieve Pass@1 competitive with 120-355B and 685B-1T parameter systems respectively, while v1.0-30B uses 25.8-95.3% fewer reasoning tokens than comparable agentic LLMs. RL increases average planning horizon by 22.8% while planning frequency grows only 2.0%, showing it learns to plan further ahead rather than more often. More broadly, learned self-regulation instantiates a principle we expect to extend beyond planning to how agents govern their own learning and adaptation.

Problem

• agentic reasoning
• planning efficiency
• self-regulation
• simulative reasoning
• token efficiency

Innovation

• agentic reasoning
• self-regulation
• simulative planning
• world model
• chain-of-thought