FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of Web Agents

2025-10-03

AI Summary

Large language model (LLM)-driven web agents face challenges in long-webpage interaction, including context saturation, high computational overhead, and susceptibility to prompt injection attacks. To address these issues, this paper proposes FocusAgent—a task-oriented, lightweight pruning method leveraging an LLM-based retriever. Its core innovation lies in semantic-guided extraction of action-relevant nodes from the accessibility tree (AxTree), enabling context compression that preserves task-critical information while ensuring semantic consistency. Experiments on WorkArena and WebArena demonstrate that FocusAgent reduces input token count by over 50% on average, maintains baseline performance, accelerates inference significantly, and decreases prompt injection success rate by 62.3%. By jointly optimizing efficiency, security, and generalizability, FocusAgent provides a lightweight and robust solution for long-context web interaction.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Web agents powered by large language models (LLMs) must process lengthy web page observations to complete user goals; these pages often exceed tens of thousands of tokens. This saturates context limits and increases computational cost processing; moreover, processing full pages exposes agents to security risks such as prompt injection. Existing pruning strategies either discard relevant content or retain irrelevant context, leading to suboptimal action prediction. We introduce FocusAgent, a simple yet effective approach that leverages a lightweight LLM retriever to extract the most relevant lines from accessibility tree (AxTree) observations, guided by task goals. By pruning noisy and irrelevant content, FocusAgent enables efficient reasoning while reducing vulnerability to injection attacks. Experiments on WorkArena and WebArena benchmarks show that FocusAgent matches the performance of strong baselines, while reducing observation size by over 50%. Furthermore, a variant of FocusAgent significantly reduces the success rate of prompt-injection attacks, including banner and pop-up attacks, while maintaining task success performance in attack-free settings. Our results highlight that targeted LLM-based retrieval is a practical and robust strategy for building web agents that are efficient, effective, and secure.

Problem

• Web agents face excessive context from lengthy web pages
• Existing pruning methods lose relevant or keep irrelevant content
• Large observations increase computational costs and security risks

Innovation

• Uses lightweight LLM retriever for content extraction
• Prunes irrelevant AxTree observations using task goals
• Reduces observation size while maintaining performance benchmarks