Retrieval-Augmented Generation by Evidence Retroactivity in LLMs

2025-01-07

AI Summary

To address error propagation and answer bias arising from unidirectional retrieval-then-reasoning in multi-hop question answering, this paper proposes RetroRAG, the first framework introducing backtracking-style reasoning. Its core is an evidence backtracking mechanism: inferring entity-centric queries to dynamically revise retrieved evidence and reconstruct reasoning paths, enabling iterative refinement and dynamic reorganization of trustworthy evidence through coordinated multi-round retrieval-generation-evaluation cycles. This establishes a closed-loop “evidence curation–discovery–verification” process, substantially enhancing robustness and interpretability for complex reasoning. On mainstream multi-hop QA benchmarks, RetroRAG consistently outperforms existing RAG methods, achieving significant gains in answer accuracy—particularly under challenging conditions involving long reasoning chains and noisy evidence.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Retrieval-augmented generation has gained significant attention due to its ability to integrate relevant external knowledge, enhancing the accuracy and reliability of the LLMs' responses. Most of the existing methods apply a dynamic multiple retrieval-generating process, to address multi-hop complex questions by decomposing them into sub-problems. However, these methods rely on an unidirectional forward reasoning paradigm, where errors from insufficient reasoning steps or inherent flaws in current retrieval systems are irreversible, potentially derailing the entire reasoning chain. For the first time, this work introduces Retroactive Retrieval-Augmented Generation (RetroRAG), a novel framework to build a retroactive reasoning paradigm. RetroRAG revises and updates the evidence, redirecting the reasoning chain to the correct direction. RetroRAG constructs an evidence-collation-discovery framework to search, generate, and refine credible evidence. It synthesizes inferential evidence related to the key entities in the question from the existing source knowledge and formulates search queries to uncover additional information. As new evidence is found, RetroRAG continually updates and organizes this information, enhancing its ability to locate further necessary evidence. Paired with an Answerer to generate and evaluate outputs, RetroRAG is capable of refining its reasoning process iteratively until a reliable answer is obtained. Empirical evaluations show that RetroRAG significantly outperforms existing methods.

Problem

• Large Language Models
• Information Retrieval
• Accuracy Improvement

Innovation

• RetroRAG
• Enhanced Information Retrieval
• Accuracy Improvement