FABLE: Forest-Based Adaptive Bi-Path LLM-Enhanced Retrieval for Multi-Document Reasoning

2026-01-26

AI Summary

Existing RAG systems suffer from semantic noise due to flat retrieval, hindering structured cross-document reasoning, while long-context large language models (LLMs) face challenges such as loss of intermediate information, high computational costs, and poor scalability. To address these limitations, this work proposes FABLE, a novel framework that deeply integrates LLMs into both retrieval structure construction and query path planning. FABLE leverages an LLM-driven, multi-granular semantic hierarchical forest index and employs a dual-path adaptive retrieval mechanism—combining LLM-guided hierarchical traversal with structure-aware propagation—to achieve fine-grained evidence retrieval while balancing efficiency. Experimental results demonstrate that FABLE achieves accuracy comparable to full-context LLMs on multi-document reasoning tasks while reducing token consumption by up to 94%.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The rapid expansion of long-context Large Language Models (LLMs) has reignited debate on whether Retrieval-Augmented Generation (RAG) remains necessary. However, empirical evidence reveals persistent limitations of long-context inference, including the lost-in-the-middle phenomenon, high computational cost, and poor scalability for multi-document reasoning. Conversely, traditional RAG systems, while efficient, are constrained by flat chunk-level retrieval that introduces semantic noise and fails to support structured cross-document synthesis. We present \textbf{FABLE}, a \textbf{F}orest-based \textbf{A}daptive \textbf{B}i-path \textbf{L}LM-\textbf{E}nhanced retrieval framework that integrates LLMs into both knowledge organization and retrieval. FABLE constructs LLM-enhanced hierarchical forest indexes with multi-granularity semantic structures, then employs a bi-path strategy combining LLM-guided hierarchical traversal with structure-aware propagation for fine-grained evidence acquisition, with explicit budget control for adaptive efficiency trade-offs. Extensive experiments demonstrate that FABLE consistently outperforms SOTA RAG methods and achieves comparable accuracy to full-context LLM inference with up to 94\% token reduction, showing that long-context LLMs amplify rather than fully replace the need for structured retrieval.

Problem

• multi-document reasoning
• retrieval-augmented generation
• long-context LLMs
• semantic noise
• structured retrieval

Innovation

• Forest-based indexing
• Bi-path retrieval
• LLM-enhanced retrieval
• Multi-granularity semantic structure
• Adaptive budget control