Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem

2026-02-21

AI Summary

This work addresses the asymmetric pipeline dependency between the reranker and generator in traditional retrieval-augmented generation (RAG) systems, which constrains overall performance. The authors propose CoRAG, a novel framework that formulates RAG as a cooperative multi-agent decision-making problem for the first time. By employing multi-agent reinforcement learning, CoRAG jointly trains the reranker and generator to collaboratively optimize toward a shared objective, thereby replacing the conventional unidirectional dependency with end-to-end coordination between retrieval and generation. Experimental results demonstrate that CoRAG, trained on only approximately 10,000 PopQA samples, significantly outperforms baseline methods in terms of generation quality, generalization capability, and stability.

Technology Category

• Multiagent Systems: Multiagent Learning
• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Retrieval-Augmented Generation (RAG) has demonstrated strong effectiveness in knowledge-intensive tasks by grounding language generation in external evidence. Despite its success, many existing RAG systems are built based on a ranking-centric, asymmetric dependency paradigm, where the generation quality of the generator is highly dependent on reranking results of the reranker. To overcome this limitation, we reformulate RAG as a cooperative multi-agent decision-making problem and propose Cooperative Retrieval-Augmented Generation (CoRAG), a framework in which the reranker and the generator act as peer decision-makers rather than being connected through an asymmetric dependency pipeline. By jointly optimizing their behaviors toward a shared task objective, the reranker and generator are encouraged to cooperate, ensuring that document reranking and generation work in concert to improve the final response. Experimental results demonstrate good generalization and improved generation stability of CoRAG, even when the model is trained on only around 10K PopQA samples. Our model released in https://anonymous.4open.science/r/CoRAG-D63F

Problem

• Retrieval-Augmented Generation
• asymmetric dependency
• reranking
• generation quality
• cooperative decision-making

Innovation

• Retrieval-Augmented Generation
• Cooperative Decision-Making
• Multi-Agent Framework
• Reranker-Generator Cooperation
• Joint Optimization