Plan-MCTS: Plan Exploration for Action Exploitation in Web Navigation

2026-02-15

AI Summary

This work addresses the challenges large language models face in web navigation, where sparse valid action paths and contextual noise impede accurate state perception. To overcome these limitations, the authors propose Plan-MCTS, a framework that shifts exploration into a semantic planning space, decoupling high-level strategic planning from low-level execution. The approach constructs a dense plan tree and leverages abstracted semantic history to enhance navigation efficiency. A dual-gated reward mechanism is introduced to jointly assess action executability and policy consistency, while structured refinement enables online repair of failed subplans. Evaluated on the WebArena benchmark, Plan-MCTS significantly improves both task success rates and search efficiency, achieving state-of-the-art performance.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Multiagent Systems: Multiagent Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Large Language Models (LLMs) have empowered autonomous agents to handle complex web navigation tasks. While recent studies integrate tree search to enhance long-horizon reasoning, applying these algorithms in web navigation faces two critical challenges: sparse valid paths that lead to inefficient exploration, and a noisy context that dilutes accurate state perception. To address this, we introduce Plan-MCTS, a framework that reformulates web navigation by shifting exploration to a semantic Plan Space. By decoupling strategic planning from execution grounding, it transforms sparse action space into a Dense Plan Tree for efficient exploration, and distills noisy contexts into an Abstracted Semantic History for precise state awareness. To ensure efficiency and robustness, Plan-MCTS incorporates a Dual-Gating Reward to strictly validate both physical executability and strategic alignment and Structural Refinement for on-policy repair of failed subplans. Extensive experiments on WebArena demonstrate that Plan-MCTS achieves state-of-the-art performance, surpassing current approaches with higher task effectiveness and search efficiency.

Problem

• web navigation
• sparse valid paths
• noisy context
• state perception
• exploration efficiency

Innovation

• Plan-MCTS
• semantic planning
• Monte Carlo Tree Search
• web navigation
• LLM agents