ELITE: Embedding-Less retrieval with Iterative Text Exploration

2025-05-17

AI Summary

To address semantic misalignment in embedding-based retrieval and the excessive computational overhead of graph-structured methods in Retrieval-Augmented Generation (RAG), this paper proposes an embedding-free, graph-structure-agnostic iterative text exploration framework. Leveraging the inherent logical reasoning capabilities of large language models (LLMs), the framework dynamically prunes the search space via importance scoring and automatically expands relevant textual passages based on inferred logical relationships. It integrates LLM-driven iterative retrieval, importance-weighted filtering, logic-guided expansion, and lightweight context re-ranking. Evaluated on long-context QA benchmarks—including NovelQA and Marathon—the method significantly outperforms strong baselines while reducing both storage and computational costs by over one order of magnitude. To our knowledge, this is the first RAG paradigm achieving high accuracy with low overhead, without reliance on embeddings or explicit graph construction.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large Language Models (LLMs) have achieved impressive progress in natural language processing, but their limited ability to retain long-term context constrains performance on document-level or multi-turn tasks. Retrieval-Augmented Generation (RAG) mitigates this by retrieving relevant information from an external corpus. However, existing RAG systems often rely on embedding-based retrieval trained on corpus-level semantic similarity, which can lead to retrieving content that is semantically similar in form but misaligned with the question's true intent. Furthermore, recent RAG variants construct graph- or hierarchy-based structures to improve retrieval accuracy, resulting in significant computation and storage overhead. In this paper, we propose an embedding-free retrieval framework. Our method leverages the logical inferencing ability of LLMs in retrieval using iterative search space refinement guided by our novel importance measure and extend our retrieval results with logically related information without explicit graph construction. Experiments on long-context QA benchmarks, including NovelQA and Marathon, show that our approach outperforms strong baselines while reducing storage and runtime by over an order of magnitude.

Problem

• LLMs struggle with long-term context retention in document-level tasks
• Existing RAG systems retrieve semantically similar but misaligned content
• Current RAG variants have high computation and storage overhead

Innovation

• Embedding-free retrieval framework using LLMs
• Iterative search space refinement with importance measure
• Logic-based retrieval without explicit graph construction