Beyond Sequential Reranking: Reranker-Guided Search Improves Reasoning Intensive Retrieval

2025-09-08

AI Summary

Traditional retrieval-then-reranking pipelines suffer from two key limitations: dependency on the quality of initial retrieval and the high computational cost of large language model (LLM)-based rerankers. To address these, we propose Reranker-Guided Search (RGS), the first approach to explicitly incorporate reranker preferences into the retrieval process. RGS constructs a proximity graph over approximate nearest neighbors, then performs greedy path search guided jointly by embedding similarity and gradients of reranker scores—dynamically prioritizing high-potential documents. This breaks the rigid “retrieve-then-rerank” sequential paradigm and enables end-to-end optimization under a fixed reranking budget (100 documents). Evaluated on BRIGHT, FollowIR, and M-BEIR benchmarks, RGS improves Recall@100 by 3.5, 2.9, and 5.1 percentage points, respectively, significantly alleviating the precision-efficiency trade-off.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Learning Preferences or Rankings
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

The widely used retrieve-and-rerank pipeline faces two critical limitations: they are constrained by the initial retrieval quality of the top-k documents, and the growing computational demands of LLM-based rerankers restrict the number of documents that can be effectively processed. We introduce Reranker-Guided-Search (RGS), a novel approach that bypasses these limitations by directly retrieving documents according to reranker preferences rather than following the traditional sequential reranking method. Our method uses a greedy search on proximity graphs generated by approximate nearest neighbor algorithms, strategically prioritizing promising documents for reranking based on document similarity. Experimental results demonstrate substantial performance improvements across multiple benchmarks: 3.5 points on BRIGHT, 2.9 on FollowIR, and 5.1 on M-BEIR, all within a constrained reranker budget of 100 documents. Our analysis suggests that, given a fixed pair of embedding and reranker models, strategically selecting documents to rerank can significantly improve retrieval accuracy under limited reranker budget.

Problem

• Overcoming initial retrieval quality constraints
• Reducing computational demands of LLM rerankers
• Improving document selection for reranking efficiency

Innovation

• Reranker-Guided-Search directly retrieves documents by reranker preferences
• Uses greedy search on proximity graphs from nearest neighbors
• Strategically prioritizes promising documents based on similarity