Reinforced Internal-External Knowledge Synergistic Reasoning for Efficient Adaptive Search Agent

2025-05-12

AI Summary

Existing LLM-based search agents over-rely on external retrieval, leading to redundant API calls, knowledge conflicts, and inference latency. To address this, we propose an adaptive search agent that— for the first time—integrates internal knowledge confidence modeling into a reinforcement learning (RL) framework. Our approach jointly optimizes retrieval decisions and knowledge fusion via a knowledge-boundary-aware reward function and a customized RAG training paradigm. The method comprises three core components: (1) a knowledge boundary identification module, (2) an RL-driven dynamic retrieval decision mechanism, and (3) an internal-external knowledge co-fusion architecture. Evaluated on multi-task knowledge reasoning benchmarks, our agent reduces retrieval frequency by over 40%, significantly improves answer accuracy, and demonstrates strong generalization across domains. This work establishes a novel paradigm for efficient, synergistic integration of parametric (internal) and retrieval-augmented (external) knowledge in LLM agents.

Technology Category

• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Retrieval-augmented generation (RAG) is a common strategy to reduce hallucinations in Large Language Models (LLMs). While reinforcement learning (RL) can enable LLMs to act as search agents by activating retrieval capabilities, existing ones often underutilize their internal knowledge. This can lead to redundant retrievals, potential harmful knowledge conflicts, and increased inference latency. To address these limitations, an efficient and adaptive search agent capable of discerning optimal retrieval timing and synergistically integrating parametric (internal) and retrieved (external) knowledge is in urgent need. This paper introduces the Reinforced Internal-External Knowledge Synergistic Reasoning Agent (IKEA), which could indentify its own knowledge boundary and prioritize the utilization of internal knowledge, resorting to external search only when internal knowledge is deemed insufficient. This is achieved using a novel knowledge-boundary aware reward function and a knowledge-boundary aware training dataset. These are designed for internal-external knowledge synergy oriented RL, incentivizing the model to deliver accurate answers, minimize unnecessary retrievals, and encourage appropriate external searches when its own knowledge is lacking. Evaluations across multiple knowledge reasoning tasks demonstrate that IKEA significantly outperforms baseline methods, reduces retrieval frequency significantly, and exhibits robust generalization capabilities.

Problem

• Optimize retrieval timing for LLMs to reduce redundant searches
• Synergize internal and external knowledge to minimize harmful conflicts
• Enhance efficiency by prioritizing internal knowledge over external retrieval

Innovation

• Reinforced synergy of internal and external knowledge
• Knowledge-boundary aware reward function for RL
• Adaptive retrieval based on internal knowledge sufficiency