Detecting Hallucinations in Retrieval-Augmented Generation via Semantic-level Internal Reasoning Graph

2026-01-06

arXiv.org

AI Summary

This work addresses the challenge of faithfulness hallucinations in retrieval-augmented generation (RAG) systems, which often arise from the opaque internal reasoning of large language models and evade detection by existing methods due to their inability to capture fine-grained semantic dependencies. To overcome this limitation, the authors propose a novel semantic-level internal reasoning graph approach that elevates reasoning graphs from the token level to the semantic level. By leveraging an enhanced layer-wise relevance propagation algorithm, the method explicitly models the model’s semantic reasoning pathways. Combined with a compact pretrained language model and a dynamic thresholding mechanism, this framework enables efficient and precise hallucination detection. Evaluated on the RAGTruth and Dolly-15k benchmarks, the proposed approach significantly outperforms state-of-the-art methods, demonstrating improved accuracy and robustness in identifying faithfulness hallucinations.

Technology Category

• Knowledge Representation and Reasoning: Reasoning with Beliefs
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The Retrieval-augmented generation (RAG) system based on Large language model (LLM) has made significant progress. It can effectively reduce factuality hallucinations, but faithfulness hallucinations still exist. Previous methods for detecting faithfulness hallucinations either neglect to capture the models'internal reasoning processes or handle those features coarsely, making it difficult for discriminators to learn. This paper proposes a semantic-level internal reasoning graph-based method for detecting faithfulness hallucination. Specifically, we first extend the layer-wise relevance propagation algorithm from the token level to the semantic level, constructing an internal reasoning graph based on attribution vectors. This provides a more faithful semantic-level representation of dependency. Furthermore, we design a general framework based on a small pre-trained language model to utilize the dependencies in LLM's reasoning for training and hallucination detection, which can dynamically adjust the pass rate of correct samples through a threshold. Experimental results demonstrate that our method achieves better overall performance compared to state-of-the-art baselines on RAGTruth and Dolly-15k.

Problem

• hallucination detection
• retrieval-augmented generation
• faithfulness hallucination
• internal reasoning
• semantic-level representation

Innovation

• semantic-level reasoning
• hallucination detection
• retrieval-augmented generation
• internal reasoning graph
• layer-wise relevance propagation