ActiveRAG: Autonomously Knowledge Assimilation and Accommodation through Retrieval-Augmented Agents

2024-02-21

AI Summary

Large language models (LLMs) suffer from noise sensitivity and conflicts between retrieved external knowledge and internal parametric knowledge, leading to inconsistent outputs and degraded performance. To address this, we propose a multi-agent Retrieval-Augmented Generation (RAG) framework featuring two novel agents: the *Knowledge Assimilation Agent*, which actively integrates retrieved knowledge with the model’s parametric memory to enable deep knowledge internalization; and the *Reasoning Calibration Agent*, which dynamically refines the inference process to enhance logical coherence and self-consistency. Inspired by human learning—specifically the stages of comprehension and adaptive adjustment—our framework effectively mitigates noise interference and knowledge conflicts. Empirical evaluation across multiple knowledge-intensive question-answering benchmarks demonstrates an approximately 10% improvement over standard RAG, alongside substantial gains in answer accuracy and self-consistency.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Multiagent Learning
• Knowledge Representation and Reasoning: Knowledge Acquisition

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Retrieval-Augmented Generation (RAG) enables Large Language Models (LLMs) to leverage external knowledge, enhancing their performance on knowledge-intensive tasks. However, existing RAG models often treat LLMs as passive recipients of information, which can lead to interference from noisy retrieved content. In this paper, we introduce ActiveRAG, a multi-agent framework that mimics human learning behavior to help LLMs actively engage with and learn from retrieved evidence. ActiveRAG designs a knowledge assimilation agent to form the knowledge understanding by associating external knowledge with the parametric memory of LLMs. Then our model employs the thought accommodation agent to calibrate the internal thought of LLMs for response refinement. Our experiments show that ActiveRAG achieves a 10% improvement over vanilla RAG on various question-answering benchmarks. Further analysis reveals that ActiveRAG mitigates the impact of noisy retrievals, alleviates conflicts between external knowledge and parametric memory and improves the self-consistency of LLMs in answering the question. All data and codes are available at https://github.com/OpenMatch/ActiveRAG.

Problem

• Enhancing LLMs' integration of unfamiliar external knowledge
• Improving consistency in knowledge utilization across tasks
• Addressing suboptimal behaviors when leveraging new information

Innovation

• Framework enhances knowledge use via constructivist modeling
• Assimilates new information to align with internal memory
• Accommodates reasoning to promote consistent reliable outputs