WebLeaper: Empowering Efficiency and Efficacy in WebAgent via Enabling Info-Rich Seeking

2025-10-28

AI Summary

Existing LLM-based information-seeking agents suffer from low search efficiency and limited generalization due to sparse target entities in knowledge sources. To address this, we propose TreeIR, a tree-structured reasoning framework that synthesizes high-quality, diverse search trajectories via three task-generation strategies—Basic, Union, and Reverse-Union—guided by Wikipedia tables to ensure broad coverage. We further introduce an efficient trajectory filtering mechanism to enhance training data quality. Evaluated on five open-domain question answering benchmarks, TreeIR achieves substantial improvements over strong baselines: average search steps decrease by 32.7%, answer accuracy increases by 11.4%, and end-to-end performance as well as cross-domain generalization both show significant gains.

Technology Category

• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Constraint Satisfaction and Optimization: Search

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Model (LLM)-based agents have emerged as a transformative approach for open-ended problem solving, with information seeking (IS) being a core capability that enables autonomous reasoning and decision-making. While prior research has largely focused on improving retrieval depth, we observe that current IS agents often suffer from low search efficiency, which in turn constrains overall performance. A key factor underlying this inefficiency is the sparsity of target entities in training tasks, which limits opportunities for agents to learn and generalize efficient search behaviors. To address these challenges, we propose WebLeaper, a framework for constructing high-coverage IS tasks and generating efficient solution trajectories. We formulate IS as a tree-structured reasoning problem, enabling a substantially larger set of target entities to be embedded within a constrained context. Leveraging curated Wikipedia tables, we propose three variants for synthesizing IS tasks, Basic, Union, and Reverse-Union, to systematically increase both IS efficiency and efficacy. Finally, we curate training trajectories by retaining only those that are simultaneously accurate and efficient, ensuring that the model is optimized for both correctness and search performance. Extensive experiments on both basic and comprehensive settings, conducted on five IS benchmarks, BrowserComp, GAIA, xbench-DeepSearch, WideSearch, and Seal-0, demonstrate that our method consistently achieves improvements in both effectiveness and efficiency over strong baselines.

Problem

• Improving search efficiency in information-seeking web agents
• Addressing sparse target entities in agent training tasks
• Enhancing both effectiveness and efficiency in web navigation

Innovation

• Tree-structured reasoning for info-rich seeking
• Synthesized tasks via Wikipedia table variants
• Training trajectories optimized for accuracy and efficiency