MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval

2025-10-31

AI Summary

Existing RAG systems rely on a single retriever and static top-k selection, limiting their capability to support corpus-level complex reasoning. Method: This paper introduces the first reinforcement learning–based multi-tool collaborative retrieval framework, which dynamically orchestrates four complementary tools—semantic search, keyword matching, conditional filtering, and result aggregation—to enable iterative, closed-loop retrieval-and-reasoning decisions. The approach employs a two-stage training pipeline: supervised fine-tuning followed by reinforcement learning, endowing the retrieval agent with autonomous tool selection and composition capabilities. Contribution/Results: Our method achieves state-of-the-art performance on GlobalQA, HotpotQA, and 2WikiMultiHopQA, significantly outperforming strong baselines across multiple corpus-level multi-hop reasoning tasks and overcoming fundamental limitations of conventional single-retriever paradigms.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Multiagent Systems: Multiagent Learning
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Large Language Models (LLMs) excel at reasoning and generation but are inherently limited by static pretraining data, resulting in factual inaccuracies and weak adaptability to new information. Retrieval-Augmented Generation (RAG) addresses this issue by grounding LLMs in external knowledge; However, the effectiveness of RAG critically depends on whether the model can adequately access relevant information. Existing RAG systems rely on a single retriever with fixed top-k selection, restricting access to a narrow and static subset of the corpus. As a result, this single-retriever paradigm has become the primary bottleneck for comprehensive external information acquisition, especially in tasks requiring corpus-level reasoning. To overcome this limitation, we propose MARAG-R1, a reinforcement-learned multi-tool RAG framework that enables LLMs to dynamically coordinate multiple retrieval mechanisms for broader and more precise information access. MARAG-R1 equips the model with four retrieval tools -- semantic search, keyword search, filtering, and aggregation -- and learns both how and when to use them through a two-stage training process: supervised fine-tuning followed by reinforcement learning. This design allows the model to interleave reasoning and retrieval, progressively gathering sufficient evidence for corpus-level synthesis. Experiments on GlobalQA, HotpotQA, and 2WikiMultiHopQA demonstrate that MARAG-R1 substantially outperforms strong baselines and achieves new state-of-the-art results in corpus-level reasoning tasks.

Problem

• Overcoming single-retriever limitations in RAG systems
• Enabling dynamic multi-tool coordination for information access
• Addressing corpus-level reasoning with reinforcement-learned retrieval

Innovation

• Reinforcement-learned multi-tool RAG framework
• Dynamically coordinates multiple retrieval mechanisms
• Learns tool usage via supervised and reinforcement learning