Knowledge-Aware Iterative Retrieval for Multi-Agent Systems

2025-03-17

AI Summary

This work addresses the lack of explicit reasoning and planning capabilities in large language models (LLMs) for multi-source information integration. We propose a dynamic knowledge caching–based multi-agent collaborative retrieval framework that decouples external retrieval from internal knowledge maintenance. Our method achieves a balance between exploratory diversity and answer accuracy through iterative, evidence-guided query rewriting, competitive/collaborative multi-agent contextual expansion, and traceable search path generation. Key contributions include: (1) the first explicit decoupling mechanism between external and internal knowledge; (2) adaptive, traceable iterative retrieval; and (3) a shared–expansion multi-agent reasoning paradigm. Evaluated on multi-step open-domain question answering, our approach significantly outperforms both single-step and conventional iterative baselines—with performance gains increasing with task difficulty—and exhibits adaptive convergence behavior, achieving superior accuracy while maintaining computational efficiency.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Multiagent Planning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We introduce a novel large language model (LLM)-driven agent framework, which iteratively refines queries and filters contextual evidence by leveraging dynamically evolving knowledge. A defining feature of the system is its decoupling of external sources from an internal knowledge cache that is progressively updated to guide both query generation and evidence selection. This design mitigates bias-reinforcement loops and enables dynamic, trackable search exploration paths, thereby optimizing the trade-off between exploring diverse information and maintaining accuracy through autonomous agent decision-making. Our approach is evaluated on a broad range of open-domain question answering benchmarks, including multi-step tasks that mirror real-world scenarios where integrating information from multiple sources is critical, especially given the vulnerabilities of LLMs that lack explicit reasoning or planning capabilities. The results show that the proposed system not only outperforms single-step baselines regardless of task difficulty but also, compared to conventional iterative retrieval methods, demonstrates pronounced advantages in complex tasks through precise evidence-based reasoning and enhanced efficiency. The proposed system supports both competitive and collaborative sharing of updated context, enabling multi-agent extension. The benefits of multi-agent configurations become especially prominent as task difficulty increases. The number of convergence steps scales with task difficulty, suggesting cost-effective scalability.

Problem

• Develops LLM-driven agent framework for iterative query refinement
• Mitigates bias-reinforcement loops in dynamic knowledge systems
• Enhances multi-agent decision-making in complex, open-domain tasks

Innovation

• LLM-driven agent framework iteratively refines queries
• Decouples external sources from internal knowledge cache
• Optimizes trade-off between information diversity and accuracy