Structure then Query: Enabling Precise Analytical Queries over Unstructured Documents

2026-08-13

AI Summary

This work addresses the challenge of enabling efficient and precise structured querying over unstructured documents, a task hindered by the limitations of existing vector retrieval methods—namely, ambiguous matching and high computational overhead. To overcome these issues, the authors propose an Annotation Index coupled with a SchemaLoop mechanism that automatically constructs hierarchical annotation schemas to transform unstructured text into structured data. They further introduce a SQL-extended query engine that integrates lightweight language models for attribute extraction and large language models for deep semantic reasoning, enhanced by a multi-stage cost-aware execution strategy and incremental index updates. Evaluated on three real-world datasets, the approach achieves an average F1 score of 0.87, substantially outperforming current methods, particularly in complex multi-hop and progressive reasoning queries.

Abstract

Unstructured documents constitute the majority of enterprise and web data. With the rapid development of large language models(LLMs), researchers have started to build data systems that analyze unstructured textual documents like operating on databases. However, because mainstream retrieval methods still relies on fuzzy matching based on vector similarity, accurately obtaining information and performing structured analysis and reasoning remains a major challenge. To address these limitations, AnnoIndex introduces two core fundamental components. The first is Annotation Index. The system uses a module called SchemaLoop to automatically create hierarchical annotation schemas from the raw corpus, and then uses lightweight language model to extract specific values. It turns scattered unstructured text into a materialized, structured index that enables low-cost filtering and querying. The annotation index avoids the black-box matching of vector similarity and amortizes attribute extraction costs from online queries to a one-time build. The second innovation is a Structured Query Engine. It compiles user questions into execution plans based on SQL extension. It first uses the Annotation Index for precise documents filtering, then gradually applies extraction operations in ascending order of cost, resorting to LLMs only for the remaining minimal fraction of the corpus that require deep semantic understanding. The extracted attributions are merged into the annotation index, reducing the cost of future queries. Experiments on three real-world datasets demonstrate that AnnoIndex consistently outperforms state-of-the-art baselines, achieving the highest average F1 score (0.87) while maintaining robust performance on complex multi-hop join and progressive reasoning queries.

Problem

• unstructured documents
• analytical queries
• structured analysis
• information extraction
• precise querying

Innovation

• Annotation Index
• Structured Query Engine
• SchemaLoop
• Cost-aware Extraction
• LLM-assisted Querying