STACKFEED: Structured Textual Actor-Critic Knowledge Base Editing with FeedBack

2024-10-14

arXiv.org

AI Summary

To address the propensity of large language models (LLMs) to generate erroneous or outdated information in low-resource or private-data settings, this paper proposes an expert-feedback-driven iterative knowledge base (KB) editing framework. Methodologically, it introduces a novel multi-Actor–single-Critic reinforcement learning architecture: each ReACT agent serves as a specialized Actor for structured-text editing, while a centralized Critic generates both target instructions and evaluation signals, enabling feedback-guided, closed-loop KB refinement. The approach tightly integrates retrieval-augmented generation (RAG), structured-text editing, and expert feedback. Experiments demonstrate substantial improvements in KB quality: on multiple low-resource and private-data benchmarks, the RAG system’s accuracy increases by up to 8%, outperforming state-of-the-art knowledge editing methods.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Models (LLMs) often generate incorrect or outdated information, especially in low-resource settings or when dealing with private data. To address this, Retrieval-Augmented Generation (RAG) uses external knowledge bases (KBs), but these can also suffer from inaccuracies. We introduce STACKFEED, a novel Structured Textual Actor-Critic Knowledge base editing with FEEDback approach that iteratively refines the KB based on expert feedback using a multi-actor, centralized critic reinforcement learning framework. Each document is assigned to an actor, modeled as a ReACT agent, which performs structured edits based on document-specific targeted instructions from a centralized critic. Experimental results show that STACKFEED significantly improves KB quality and RAG system performance, enhancing accuracy by up to 8% over baselines.

Problem

• Corrects inaccurate knowledge bases using expert feedback
• Improves retrieval-augmented generation system performance
• Addresses outdated information in low-resource settings

Innovation

• Iteratively refines knowledge bases using expert feedback
• Uses multi-actor centralized critic reinforcement learning framework
• Performs structured edits with document-specific ReACT agents