From Ranking to Selection: A Simple but Efficient Dynamic Passage Selector for Retrieval Augmented Generation

2025-08-13

AI Summary

Existing RAG rerankers employ fixed-top-K passage selection, which struggles with multi-hop queries requiring cross-document evidence aggregation: overly small K risks missing critical information, while overly large K introduces noise. This work proposes the Dynamic Passage Selector (DPS), the first approach to formulate passage selection as a supervised dynamic set selection task. DPS explicitly models contextual dependencies among passages and enables adaptive selection of variable-length, high-quality evidence subsets. It is plug-and-play—requiring no modification to the underlying RAG pipeline. Evaluated across five benchmarks, DPS consistently outperforms state-of-the-art rerankers. On MuSiQue, it achieves an absolute F1 improvement of 30.06% over Qwen3-reranker and 15.4% over RankingGPT, demonstrating substantial gains in multi-document reasoning capability.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Learning Preferences or Rankings
• Knowledge Representation and Reasoning: Preferences

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Retrieval-augmented generation (RAG) systems are often bottlenecked by their reranking modules, which typically score passages independently and select a fixed Top-K size. This approach struggles with complex multi-hop queries that require synthesizing evidence across multiple documents, creating a trade-off where small K values omit crucial information and large K values introduce noise. To address this, we introduce the Dynamic Passage Selector (DPS), a novel reranking framework that treats passage selection as a supervised learning problem. Unlike traditional point-wise or list-wise methods, DPS is fine-tuned to capture inter-passage dependencies and dynamically select the most relevant set of passages for generation. As a seamless plug-and-play module, DPS requires no modifications to the standard RAG pipeline. Comprehensive evaluations on five benchmarks show that DPS consistently outperforms state-of-the-art rerankers and fine-tuning methods. Notably, on the challenging MuSiQue dataset, DPS improves the F1-score by 30.06% and 15.4% over strong baselines like Qwen3-reranker and RankingGPT, respectively. Our results demonstrate that by enabling adaptive evidence selection, DPS substantially enhances reasoning capabilities in complex RAG scenarios.

Problem

• Addressing retrieval bottlenecks in RAG systems from fixed Top-K selection
• Solving complex multi-hop queries requiring evidence synthesis across documents
• Reducing noise while preventing crucial information omission in passage selection

Innovation

• Dynamic selection of passages using supervised learning
• Captures inter-passage dependencies for better relevance
• Plug-and-play module enhancing RAG without pipeline changes