🤖 AI Summary
Updating knowledge in large language models is costly, and existing retrieval-augmented approaches exhibit instability in long-context and multi-hop reasoning scenarios. This work proposes “Knowledge Capsules”—structured, non-parametric memory units—and introduces an external Key-Value Injection (KVI) framework that directly integrates external knowledge into the model’s attention mechanism rather than merely appending it as additional context. By elevating knowledge integration from the contextual level to the memory level, this approach enables efficient and stable knowledge injection while keeping the base model parameters frozen. Experimental results demonstrate that the method significantly outperforms both RAG and GraphRAG across multiple question-answering benchmarks, achieving notably higher accuracy and robustness, particularly in tasks involving long contexts and multi-hop reasoning.
📝 Abstract
Large language models (LLMs) encode knowledge in parametric weights, making it costly to update or extend without retraining. Retrieval-augmented generation (RAG) mitigates this limitation by appending retrieved text to the input, but operates purely through context expansion, where external knowledge competes as tokens within the attention mechanism. As a result, its influence is indirect and often unstable, particularly in long context and multi hop reasoning scenarios. We propose Knowledge Capsules, structured nonparametric memory units that represent normalized relational knowledge and can be constructed directly from document corpora using a frozen base model. Instead of injecting knowledge as text, we introduce an External Key Value Injection (KVI) framework that compiles capsules into attention-compatible key value representations, enabling external knowledge to directly participate in the model's attention computation. By shifting knowledge integration from context-level augmentation to memory level interaction, the proposed framework consistently outperforms RAG and GraphRAG across multiple QA benchmarks, with improved stability and accuracy in long context and multi hop reasoning, while requiring no parameter updates.