SODIUM: From Open Web Data to Queryable Databases

2026-03-18

AI Summary

This work addresses the challenge faced by domain experts in manually synthesizing information from the open web to answer complex analytical questions—a process that is often inefficient and labor-intensive. To automate this, the authors propose treating the open web as a latent database and introduce a novel task paradigm termed SODIUM, along with the first benchmark for this task, SODIUM-Bench. They develop SODIUM-Agent, a multi-agent system comprising a web explorer and a cache manager, which integrates an ATP-BFS exploration algorithm with structure-aware information extraction and caching strategies to automatically construct a queryable structured database. Evaluated on SODIUM-Bench, their approach achieves an accuracy of 91.1%, nearly doubling the performance of the strongest baseline and improving upon the weakest baseline by up to 73 times.

Technology Category

• Data Mining & Knowledge Management: Knowledge Acquisition from the Web
• Natural Language Processing: Information Extraction
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

During research, domain experts often ask analytical questions whose answers require integrating data from a wide range of web sources. Thus, they must spend substantial effort searching, extracting, and organizing raw data before analysis can begin. We formalize this process as the SODIUM task, where we conceptualize open domains such as the web as latent databases that must be systematically instantiated to support downstream querying. Solving SODIUM requires (1) conducting in-depth and specialized exploration of the open web, which is further strengthened by (2) exploiting structural correlations for systematic information extraction and (3) integrating collected information into coherent, queryable database instances. To quantify the challenges in automating SODIUM, we construct SODIUM-Bench, a benchmark of 105 tasks derived from published academic papers across 6 domains, where systems are tasked with exploring the open web to collect and aggregate data from diverse sources into structured tables. Existing systems struggle with SODIUM tasks: we evaluate 6 advanced AI agents on SODIUM-Bench, with the strongest baseline achieving only 46.5% accuracy. To bridge this gap, we develop SODIUM-Agent, a multi-agent system composed of a web explorer and a cache manager. Powered by our proposed ATP-BFS algorithm and optimized through principled management of cached sources and navigation paths, SODIUM-Agent conducts deep and comprehensive web exploration and performs structurally coherent information extraction. SODIUM-Agent achieves 91.1% accuracy on SODIUM-Bench, outperforming the strongest baseline by approximately 2 times and the weakest by up to 73 times.

Problem

• open web data
• data integration
• queryable databases
• information extraction
• structured data

Innovation

• SODIUM
• multi-agent system
• ATP-BFS algorithm
• structured data extraction
• web exploration