DeepRefine: Agent-Compiled Knowledge Refinement via Reinforcement Learning

2026-05-11

AI Summary

Existing agent knowledge bases often suffer from incompleteness, inaccuracies, and redundancy, which degrade retrieval quality and downstream task performance over iterative use. This work proposes the first end-to-end, reinforcement learning–based framework for autonomous knowledge base refinement, leveraging multi-turn interactive reasoning and abductive diagnosis to identify deficiencies and perform targeted incremental updates. The core innovation lies in a novel unsupervised reward mechanism—Gain-Beyond-Draft—that operates without gold-standard references, enabling large language models to iteratively enhance knowledge base quality. Experimental results demonstrate that the proposed approach significantly outperforms strong baselines across multiple downstream tasks, effectively improving both knowledge base fidelity and task performance.

Abstract

Agent-compiled knowledge bases provide persistent external knowledge for large language model (LLM) agents in open-ended, knowledge-intensive downstream tasks. Yet their quality is systematically limited by \emph{incompleteness}, \emph{incorrectness}, and \emph{redundancy}, manifested as missing evidence or cross-document links, low-confidence or imprecise claims, and ambiguous or coreference resolution issues. Such defects compound under iterative use, degrading retrieval fidelity and downstream task performance. We present \textbf{DeepRefine}, a general LLM-based reasoning model for \emph{agent-compiled knowledge refinement} that improves the quality of any pre-constructed knowledge bases with user queries to make it more suitable for the downstream tasks. DeepRefine performs multi-turn interactions with the knowledge base and conducts abductive diagnosis over interaction history, localizes likely defects, and executes targeted refinement actions for incremental knowledge base updates. To optimize refinement policies of DeepRefine without gold references, we introduce a Gain-Beyond-Draft (GBD) reward and train the reasoning process end-to-end via reinforcement learning. Extensive experiments demonstrate consistent downstream gains over strong baselines.

Problem

• incompleteness
• incorrectness
• redundancy
• knowledge base refinement
• agent-compiled knowledge

Innovation

• knowledge refinement
• reinforcement learning
• agent-compiled knowledge
• abductive diagnosis
• Gain-Beyond-Draft