Region4Web: Rethinking Observation Space Granularity for Web Agents

2026-05-07

AI Summary

This work addresses the limitation of existing web agents that observe web pages at the element-level granularity, thereby neglecting the underlying functional structure and requiring repeated semantic inference at each step. To overcome this, the authors propose elevating the observation granularity to functional regions by hierarchically decomposing and semantically abstracting the accessibility tree (AXTree). They introduce a PageDigest mechanism that generates compact, step-consistent page summaries. For the first time, functional regions are explicitly modeled as the fundamental observational units for web agents, capturing the page’s functional layout. Evaluated on the WebArena benchmark, this approach significantly reduces observation length and consistently improves task success rates across diverse large language models and agent architectures, demonstrating the effectiveness and generality of region-level observations.

Abstract

Web agents perceive web pages through an observation space, yet its granularity has remained an underexamined design choice. Existing work treats observation at the same element-level granularity as the action space, leaving the page's functional organization implicit and forcing the agent to infer it from element-level signals at every step. We argue observation should instead operate at the granularity of functional regions, parts of the page that each serve a distinct purpose. We propose Region4Web, a framework that reorganizes the AXTree into functional regions through hierarchical decomposition and semantic abstraction, exposing the page's functional organization as the basis for page state understanding. Moreover, we propose PageDigest, a web-specific inference pipeline that delivers this region-level observation to the actor agent as a compact per-page digest that persists across steps. On the WebArena benchmark, PageDigest substantially reduces observation length while improving overall task success rate across diverse backbone large language models (LLMs) and established agent methods, regardless of backbone capacity. These results show that operating at the granularity of functional regions delivers a more compact and informative basis for the actor agent than element-level processing alone.

Problem

• observation space granularity
• web agents
• functional regions
• page understanding
• element-level processing

Innovation

• functional regions
• observation space granularity
• Region4Web
• PageDigest
• web agents