Compositional Monte Carlo Tree Diffusion for Extendable Planning

2025-10-24

AI Summary

Existing Monte Carlo Tree Diffusion (MCTD) methods are constrained by fixed training trajectory lengths, supporting only single-trajectory local search without global planning capability. To address this limitation, we propose Compositional MCTD—a novel framework that for the first time formalizes planning as compositional reasoning across trajectory segments, implemented via three types of combiners: online, distributed, and pre-planning. Our approach integrates diffusion models with Monte Carlo tree search, incorporating parallel exploration, plan-graph caching, and global search strategies to enable efficient, scalable sequential decision-making for long-horizon tasks. Experiments demonstrate significant improvements in success rates and trajectory coherence on long-range tasks, alongside markedly enhanced inference efficiency.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Multiagent Systems: Multiagent Planning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Monte Carlo Tree Diffusion (MCTD) integrates diffusion models with structured tree search to enable effective trajectory exploration through stepwise reasoning. However, MCTD remains fundamentally limited by training trajectory lengths. While periodic replanning allows plan concatenation for longer plan generation, the planning process remains locally confined, as MCTD searches within individual trajectories without access to global context. We propose Compositional Monte Carlo Tree Diffusion (C-MCTD), a framework that elevates planning from individual trajectory optimization to reasoning over complete plan compositions. C-MCTD introduces three complementary components: (1) Online Composer, which performs globally-aware planning by searching across entire plan compositions; (2) Distributed Composer, which reduces search complexity through parallel exploration from multiple starting points; and (3) Preplan Composer, which accelerates inference by leveraging cached plan graphs.

Problem

• Extends planning beyond training trajectory length limitations
• Enables globally-aware reasoning across complete plan compositions
• Reduces search complexity through parallel exploration techniques

Innovation

• Globally-aware planning across entire plan compositions
• Parallel exploration from multiple starting points
• Accelerated inference using cached plan graphs