LLM-guided Hierarchical Retrieval

2025-10-15

AI Summary

Existing retrieval paradigms face fundamental limitations for complex, multi-faceted queries requiring deep reasoning: embedding-based retrieval suffers from shallow semantic understanding; generative approaches lack efficient update mechanisms; and long-context LLM-based retrieval is computationally infeasible. To address this, we propose LATTICE—a hierarchical retrieval framework for large-scale document collections. LATTICE constructs a multi-level semantic summary tree via unsupervised, LLM-driven semantic clustering (top-down or bottom-up), integrating offline indexing with online LLM-guided tree traversal. We further introduce a global relevance scoring algorithm with local output calibration to mitigate LLM-induced noise and cross-level score incomparability. Crucially, LATTICE requires no fine-tuning. On the BRIGHT benchmark under zero-shot evaluation, it achieves state-of-the-art performance: +9% Recall@100 and +5% nDCG@10 over prior methods—matching the performance of the fine-tuned DIVER-v2 while eliminating training overhead.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Summarization

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Modern IR systems are increasingly tasked with answering complex, multi-faceted queries that require deep reasoning rather than simple keyword or semantic matching. While LLM-based IR has shown great promise, the prevailing retrieve-then-rerank paradigm inherits the limitations of embedding-based retrieval; parametric generative approaches are difficult to update with new information; and long-context methods that place the entire corpus in context are computationally infeasible for large document collections. To address these challenges, we introduce LATTICE, a hierarchical retrieval framework that enables an LLM to reason over and navigate large corpora with logarithmic search complexity by imposing a semantic tree structure on the corpus. Our approach consists of two stages: (1) an offline phase that organizes the corpus into a semantic hierarchy via either a bottom-up agglomerative strategy or a top-down divisive strategy using multi-level summaries and (2) an online traversal phase where a search LLM navigates this tree. A central challenge in such LLM-guided search is that the model's relevance judgments are noisy, context-dependent, and unaware of the hierarchy, making cross-branch and cross-level comparisons difficult. To overcome this, we propose a traversal algorithm that estimates calibrated latent relevance scores from local LLM outputs and aggregates them into a global path relevance metric. Our training-free framework achieves state-of-the-art zero-shot performance on the reasoning-intensive BRIGHT benchmark, demonstrating up to 9% improvement in Recall@100 and 5% in nDCG@10 over the next best zero-shot baseline. Furthermore, compared to the fine-tuned SOTA method DIVER-v2, LATTICE attains comparable results on BRIGHT subsets that use a static corpus for evaluation.

Problem

• Addresses limitations of embedding-based retrieval in complex queries
• Enables LLMs to navigate large corpora with logarithmic search complexity
• Overcomes noisy relevance judgments in hierarchical semantic structures

Innovation

• Hierarchical retrieval framework with semantic tree structure
• Offline corpus organization using multi-level summary strategies
• Training-free traversal algorithm with calibrated relevance estimation