Context-Efficient Retrieval with Factual Decomposition

2025-03-25

AI Summary

To address the low retrieval efficiency and limited question-answering accuracy of large language models (LLMs) under constrained retrieval context windows, this paper proposes a fact-decomposition-based preprocessing paradigm for retrieval. It dynamically decomposes raw text into semi-structured atomic facts to construct a lightweight, indexable episodic memory. The method integrates fact extraction and structured representation, sparse vector retrieval, context-aware re-ranking, and atomic-fact indexing. Evaluated on multi-source question answering, it achieves a 12.7% average accuracy gain, a 38% reduction in inference latency, and a 65% decrease in context token consumption—under identical retrieval token budgets. This work is the first to introduce fact decomposition into retrieval preprocessing, significantly enhancing the precision, efficiency, and contextual token utilization of LLMs’ external retrieval.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Question Answering

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

There has recently been considerable interest in incorporating information retrieval into large language models (LLMs). Retrieval from a dynamically expanding external corpus of text allows a model to incorporate current events and can be viewed as a form of episodic memory. Here we demonstrate that pre-processing the external corpus into semi-structured ''atomic facts'' makes retrieval more efficient. More specifically, we demonstrate that our particular form of atomic facts improves performance on various question answering tasks when the amount of retrieved text is limited. Limiting the amount of retrieval reduces the size of the context and improves inference efficiency.

Problem

• Improving retrieval efficiency in large language models
• Enhancing question answering with limited retrieved text
• Pre-processing text into atomic facts for better performance

Innovation

• Pre-process corpus into semi-structured atomic facts
• Improves retrieval efficiency with limited text
• Enhances question answering performance effectively