Test-time Corpus Feedback: From Retrieval to RAG

2025-08-21

AI Summary

Existing RAG frameworks statically decouple retrieval from generation, performing single-shot retrieval before answer generation—limiting their effectiveness on complex tasks requiring iterative evidence gathering or high-precision retrieval. To address this, we propose a test-time corpus feedback-driven dynamic RAG paradigm. We introduce a dual-dimensional role taxonomy for retrievers, grounded in feedback sources and functional objectives, modeling retrieval as a learnable, self-adaptive, and evolving module. Our framework integrates iterative evidence collection, query reformulation, and context-aware re-ranking to enable closed-loop, bidirectional interaction between retrieval and generation. By unifying cross-domain feedback mechanisms, it significantly improves retrieval accuracy and end-to-end performance on knowledge-intensive tasks—including multi-hop question answering and fact verification—thereby bridging the methodological gap between information retrieval and NLP in dynamic, collaborative modeling.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Retrieval-Augmented Generation (RAG) has emerged as a standard framework for knowledge-intensive NLP tasks, combining large language models (LLMs) with document retrieval from external corpora. Despite its widespread use, most RAG pipelines continue to treat retrieval and reasoning as isolated components, retrieving documents once and then generating answers without further interaction. This static design often limits performance on complex tasks that require iterative evidence gathering or high-precision retrieval. Recent work in both the information retrieval (IR) and NLP communities has begun to close this gap by introducing adaptive retrieval and ranking methods that incorporate feedback. In this survey, we present a structured overview of advanced retrieval and ranking mechanisms that integrate such feedback. We categorize feedback signals based on their source and role in improving the query, retrieved context, or document pool. By consolidating these developments, we aim to bridge IR and NLP perspectives and highlight retrieval as a dynamic, learnable component of end-to-end RAG systems.

Problem

• Addressing static retrieval in RAG systems limiting complex task performance
• Integrating feedback mechanisms to improve retrieval and ranking dynamically
• Bridging the gap between isolated retrieval and reasoning components

Innovation

• Dynamic retrieval with feedback integration
• Adaptive ranking using contextual signals
• End-to-end learnable RAG systems