LineRetriever: Planning-Aware Observation Reduction for Web Agents

2025-06-30

AI Summary

Large language models (LLMs) face context-length limitations and degraded state/action history retention when processing lengthy DOM/AxTree observations during web navigation. To address this, we propose a planning-aware observation pruning method that dynamically assesses each element’s contribution to subsequent action prediction—without relying on semantic similarity—and retains only those components most relevant to the current planning goal. By explicitly anchoring retrieval objectives to the action prediction task, our approach jointly optimizes observation compression and state awareness. Experiments demonstrate that the method maintains navigation performance while significantly reducing per-step input length (average compression rate: 62%), effectively alleviating the context bottleneck and enhancing the agent’s ability to model both page state and historical actions.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Multiagent Planning

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

While large language models have demonstrated impressive capabilities in web navigation tasks, the extensive context of web pages, often represented as DOM or Accessibility Tree (AxTree) structures, frequently exceeds model context limits. Current approaches like bottom-up truncation or embedding-based retrieval lose critical information about page state and action history. This is particularly problematic for adaptive planning in web agents, where understanding the current state is essential for determining future actions. We hypothesize that embedding models lack sufficient capacity to capture plan-relevant information, especially when retrieving content that supports future action prediction. This raises a fundamental question: how can retrieval methods be optimized for adaptive planning in web navigation tasks? In response, we introduce extit{LineRetriever}, a novel approach that leverages a language model to identify and retrieve observation lines most relevant to future navigation steps. Unlike traditional retrieval methods that focus solely on semantic similarity, extit{LineRetriever} explicitly considers the planning horizon, prioritizing elements that contribute to action prediction. Our experiments demonstrate that extit{LineRetriever} can reduce the size of the observation at each step for the web agent while maintaining consistent performance within the context limitations.

Problem

• Reducing web page context size for model limits
• Preserving critical state and action history information
• Optimizing retrieval for adaptive planning in navigation

Innovation

• Leverages language model for relevant observation retrieval
• Prioritizes elements aiding future action prediction
• Reduces observation size while maintaining performance