Query Decomposition for RAG: Balancing Exploration-Exploitation

2025-10-21

AI Summary

To address the trade-off between retrieval breadth and precision in RAG systems for complex queries, this paper proposes a dynamic subquery selection method grounded in an exploration-exploitation mechanism. It formalizes query decomposition and document retrieval as a multi-armed bandit problem, jointly leveraging relevance ranking and human feedback signals to estimate the information gain of each subquery in real time and optimize its retrieval priority. By suppressing redundant retrievals and noise interference, the approach ensures comprehensive coverage of critical evidence while substantially improving retrieval efficiency. Experiments demonstrate a 35% improvement in document-level precision and a 15% gain in α-nDCG, alongside enhanced accuracy and coherence in long-text generation. The core contribution lies in the first integration of human judgments into subquery utility modeling, enabling online, adaptive optimization of retrieval strategies.

Technology Category

• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Intelligent Query Processing
• Machine Learning: Learning Preferences or Rankings

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Retrieval-augmented generation (RAG) systems address complex user requests by decomposing them into subqueries, retrieving potentially relevant documents for each, and then aggregating them to generate an answer. Efficiently selecting informative documents requires balancing a key trade-off: (i) retrieving broadly enough to capture all the relevant material, and (ii) limiting retrieval to avoid excessive noise and computational cost. We formulate query decomposition and document retrieval in an exploitation-exploration setting, where retrieving one document at a time builds a belief about the utility of a given sub-query and informs the decision to continue exploiting or exploring an alternative. We experiment with a variety of bandit learning methods and demonstrate their effectiveness in dynamically selecting the most informative sub-queries. Our main finding is that estimating document relevance using rank information and human judgments yields a 35% gain in document-level precision, 15% increase in α-nDCG, and better performance on the downstream task of long-form generation.

Problem

• Balancing exploration-exploitation trade-off in RAG retrieval
• Dynamically selecting informative sub-queries using bandit methods
• Improving document relevance estimation with rank information

Innovation

• Balancing exploration-exploitation trade-off in retrieval
• Using bandit learning for dynamic sub-query selection
• Leveraging rank information and human relevance judgments