Know3-RAG: A Knowledge-aware RAG Framework with Adaptive Retrieval, Generation, and Filtering

2025-05-19

AI Summary

To address the pervasive hallucination problem in large language model (LLM) generation and the declining factual consistency of retrieval-augmented generation (RAG) systems—caused by insufficient supervision and inaccurate citation—this paper proposes a knowledge graph (KG)-driven three-stage RAG framework. First, KG embedding-guided adaptive retrieval decision-making improves retrieval precision. Second, KG-enhanced query expansion generation broadens semantic coverage. Third, semantics-aligned and confidence-aware, knowledge-driven reference filtering suppresses irrelevant or erroneous citations. The method integrates KG embeddings, entity linking, semantic similarity modeling, and confidence calibration. Evaluated on multiple open-domain question-answering benchmarks, the framework significantly reduces hallucination rates while improving answer factual accuracy and reference relevance, consistently outperforming state-of-the-art RAG baselines.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recent advances in large language models (LLMs) have led to impressive progress in natural language generation, yet their tendency to produce hallucinated or unsubstantiated content remains a critical concern. To improve factual reliability, Retrieval-Augmented Generation (RAG) integrates external knowledge during inference. However, existing RAG systems face two major limitations: (1) unreliable adaptive control due to limited external knowledge supervision, and (2) hallucinations caused by inaccurate or irrelevant references. To address these issues, we propose Know3-RAG, a knowledge-aware RAG framework that leverages structured knowledge from knowledge graphs (KGs) to guide three core stages of the RAG process, including retrieval, generation, and filtering. Specifically, we introduce a knowledge-aware adaptive retrieval module that employs KG embedding to assess the confidence of the generated answer and determine retrieval necessity, a knowledge-enhanced reference generation strategy that enriches queries with KG-derived entities to improve generated reference relevance, and a knowledge-driven reference filtering mechanism that ensures semantic alignment and factual accuracy of references. Experiments on multiple open-domain QA benchmarks demonstrate that Know3-RAG consistently outperforms strong baselines, significantly reducing hallucinations and enhancing answer reliability.

Problem

• Improving factual reliability in LLMs by integrating external knowledge
• Addressing unreliable adaptive control in RAG systems with KG supervision
• Reducing hallucinations via knowledge-aware retrieval, generation, and filtering

Innovation

• KG embedding for adaptive retrieval control
• KG-derived entities enhance reference generation
• Knowledge-driven filtering ensures factual accuracy