Zero-RAG: Towards Retrieval-Augmented Generation with Zero Redundant Knowledge

2025-11-01

AI Summary

Retrieval-augmented generation (RAG) suffers from growing redundancy between large language models’ (LLMs’) internal knowledge and external knowledge bases, leading to unnecessary indexing, retrieval overhead, and potential interference with LLMs’ autonomous reasoning. Method: We propose a knowledge-aware retrieval optimization framework comprising three components: (1) a Mastery-Score metric quantifying the LLM’s intrinsic knowledge coverage over corpus segments; (2) a dynamic Query Router that determines whether external retrieval is necessary for each query; and (3) noise-tolerant fine-tuning to enhance model robustness after knowledge base pruning. Contribution/Results: Evaluated on Wikipedia, our method achieves 30% corpus pruning while improving retrieval speed by 22% and preserving generation quality. This work constitutes the first systematic modeling and elimination of knowledge redundancy in RAG, substantially reducing indexing and retrieval costs without compromising output fidelity—establishing a new paradigm for efficient, trustworthy RAG deployment.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Search and Optimization: Learning to Search
• Knowledge Representation and Reasoning: Knowledge Acquisition

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Retrieval-Augmented Generation has shown remarkable results to address Large Language Models' hallucinations, which usually uses a large external corpus to supplement knowledge to LLMs. However, with the development of LLMs, the internal knowledge of LLMs has expanded significantly, thus causing significant knowledge redundancy between the external corpus and LLMs. On the one hand, the indexing cost of dense retrieval is highly related to the corpus size and thus significant redundant knowledge intensifies the dense retrieval's workload. On the other hand, the redundant knowledge in the external corpus is not helpful to LLMs and our exploratory analysis shows that it instead hurts the RAG performance on those questions which the LLM can answer by itself. To address these issues, we propose Zero-RAG to tackle these challenges. Specifically, we first propose the Mastery-Score metric to identify redundant knowledge in the RAG corpus to prune it. After pruning, answers to "mastered" questions rely primarily on internal knowledge of the LLM. To better harness the internal capacity, we propose Query Router and Noise-Tolerant Tuning to avoid the irrelevant documents' distraction and thus further improve the LLM's utilization of internal knowledge with pruned corpus. Experimental results show that Zero-RAG prunes the Wikipedia corpus by 30% and accelerates the retrieval stage by 22%, without compromising RAG's performance.

Problem

• Reducing knowledge redundancy between external corpus and LLMs in RAG systems
• Minimizing dense retrieval workload by pruning redundant external knowledge
• Improving LLM utilization of internal knowledge with pruned corpus

Innovation

• Prunes redundant knowledge using Mastery-Score metric
• Routes queries to avoid irrelevant document distraction
• Tunes model for noise tolerance with pruned corpus