LiveWeb-IE: A Benchmark For Online Web Information Extraction

2026-03-14

AI Summary

Existing evaluations of web information extraction predominantly rely on static HTML snapshots, which fail to capture real-world performance on dynamic websites. To address this limitation, this work introduces the first online benchmark for real-time evaluation on live dynamic sites, featuring natural language queries authorized by participating websites and structured into four levels of task complexity. The paper further proposes Visual-Guided Agent (VGS), a novel framework that integrates visual perception with multi-stage reasoning to emulate human-like cognitive processes for precise information localization and extraction. Experimental results demonstrate that VGS consistently enhances extraction performance, practicality, and generalization across diverse backbone models in dynamic web environments.

Technology Category

• Natural Language Processing: Information Extraction
• Data Mining & Knowledge Management: Knowledge Acquisition from the Web
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data visualization

Abstract

Web information extraction (WIE) is the task of automatically extracting data from web pages, offering high utility for various applications. The evaluation of WIE systems has traditionally relied on benchmarks built from HTML snapshots captured at a single point in time. However, this offline evaluation paradigm fails to account for the temporally evolving nature of the web; consequently, performance on these static benchmarks often fails to generalize to dynamic real-world scenarios. To bridge this gap, we introduce \dataset, a new benchmark designed for evaluating WIE systems directly against live websites. Based on trusted and permission-granted websites, we curate natural language queries that require information extraction of various data categories, such as text, images, and hyperlinks. We further design these queries to represent four levels of complexity, based on the number and cardinality of attributes to be extracted, enabling a granular assessment of WIE systems. In addition, we propose Visual Grounding Scraper (VGS), a novel multi-stage agentic framework that mimics human cognitive processes by visually narrowing down web page content to extract desired information. Extensive experiments across diverse backbone models demonstrate the effectiveness and robustness of VGS. We believe that this study lays the foundation for developing practical and robust WIE systems.

Problem

• Web Information Extraction
• Live Web Evaluation
• Dynamic Web Content
• Benchmarking
• Temporal Evolution

Innovation

• LiveWeb-IE
• online web information extraction
• Visual Grounding Scraper
• dynamic web evaluation
• agentic framework