Better wit than wealth: Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement

2025-03-31

AI Summary

To address the high inference cost and frequent hallucinations of conventional RAG, as well as the substantial training/storage overhead and poor generalization of parameterized RAG (PRAG), this paper proposes DyPRAG—a dynamic parametric RAG framework. Its core is a lightweight parametric translator that, at inference time, maps external documents on-the-fly into LLM-fusible parametric knowledge, enabling dynamic parameter updates and knowledge conflict resolution. Crucially, DyPRAG requires no fine-tuning of the backbone LLM, drastically reducing inference, training, and storage costs. Extensive evaluation across multiple benchmarks demonstrates its effectiveness in suppressing hallucinations, enhancing knowledge fusion quality, and consistently outperforming state-of-the-art RAG and PRAG methods. DyPRAG introduces the novel “inference-time dynamic parametrization” paradigm, achieving a principled balance among efficiency, generalization, and practical deployability.

Technology Category

• Natural Language Processing: Discourse, Pragmatics & Argument Mining
• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Intelligent Query Processing

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Retrieval-augmented generation (RAG) enhances large language models (LLMs) by retrieving relevant documents from external sources and incorporating them into the context. While it improves reliability by providing factual texts, it significantly increases inference costs as context length grows and introduces challenging issue of RAG hallucination, primarily caused by the lack of corresponding parametric knowledge in LLMs. An efficient solution is to enhance the knowledge of LLMs at test-time. Parametric RAG (PRAG) addresses this by embedding document into LLMs parameters to perform test-time knowledge enhancement, effectively reducing inference costs through offline training. However, its high training and storage costs, along with limited generalization ability, significantly restrict its practical adoption. To address these challenges, we propose Dynamic Parametric RAG (DyPRAG), a novel framework that leverages a lightweight parameter translator model to efficiently convert documents into parametric knowledge. DyPRAG not only reduces inference, training, and storage costs but also dynamically generates parametric knowledge, seamlessly enhancing the knowledge of LLMs and resolving knowledge conflicts in a plug-and-play manner at test-time. Extensive experiments on multiple datasets demonstrate the effectiveness and generalization capabilities of DyPRAG, offering a powerful and practical RAG paradigm which enables superior knowledge fusion and mitigates RAG hallucination in real-world applications. Our code is available at https://github.com/Trae1ounG/DyPRAG.

Problem

• Reducing inference costs in retrieval-augmented generation (RAG) systems
• Mitigating RAG hallucination caused by lack of parametric knowledge
• Enhancing LLMs' knowledge dynamically and efficiently at test-time

Innovation

• Dynamic Parametric RAG for test-time enhancement
• Lightweight translator reduces costs dynamically
• Plug-and-play knowledge fusion mitigates hallucination