Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers

2025-12-11

AI Summary

To address frequent retrieval errors and hallucinations in Retrieval-Augmented Generation (RAG) for both multi-hop and single-hop question answering, this paper proposes CoopRAG—a cooperative RAG framework. CoopRAG guides precise retrieval via reasoning-chain generation with sub-question decomposition and uncertainty-aware token masking; introduces the first intra-retriever multi-layer contrastive learning mechanism for fine-grained document re-ranking; and establishes a bidirectional knowledge exchange paradigm between the retriever and LLM, where the LLM dynamically fills masked reasoning chains to generate answers. Key innovations include: (1) inter-layer contrastive re-ranking, (2) masked dynamic reasoning-chain modeling, and (3) a cooperative RAG architecture. Extensive experiments on three multi-hop QA benchmarks and one single-hop QA dataset demonstrate consistent superiority over state-of-the-art methods, achieving significant improvements in retrieval accuracy and answer factual consistency.

Technology Category

• Natural Language Processing: Question Answering
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Since large language models (LLMs) have a tendency to generate factually inaccurate output, retrieval-augmented generation (RAG) has gained significant attention as a key means to mitigate this downside of harnessing only LLMs. However, existing RAG methods for simple and multi-hop question answering (QA) are still prone to incorrect retrievals and hallucinations. To address these limitations, we propose CoopRAG, a novel RAG framework for the question answering task in which a retriever and an LLM work cooperatively with each other by exchanging informative knowledge, and the earlier and later layers of the retriever model work cooperatively with each other to accurately rank the retrieved documents relevant to a given query. In this framework, we (i) unroll a question into sub-questions and a reasoning chain in which uncertain positions are masked, (ii) retrieve the documents relevant to the question augmented with the sub-questions and the reasoning chain, (iii) rerank the documents by contrasting layers of the retriever, and (iv) reconstruct the reasoning chain by filling the masked positions via the LLM. Our experiments demonstrate that CoopRAG consistently outperforms state-of-the-art QA methods on three multi-hop QA datasets as well as a simple QA dataset in terms of both the retrieval and QA performances. Our code is available.footnote{https://github.com/meaningful96/CoopRAG}

Problem

• Addresses inaccurate retrievals and hallucinations in existing RAG methods for QA
• Proposes cooperative retriever-LLM interaction via knowledge exchange and sub-question unrolling
• Enhances document ranking by contrasting retriever layers and reconstructing reasoning chains

Innovation

• Cooperative retriever-LLM framework exchanges knowledge for accurate QA.
• Unrolls questions into sub-questions and masked reasoning chains for retrieval.
• Reranks documents by contrasting retriever layers to improve relevance.