LRP4RAG: Detecting Hallucinations in Retrieval-Augmented Generation via Layer-wise Relevance Propagation

2024-08-28

arXiv.org

AI Summary

To address hallucination in retrieval-augmented generation (RAG) caused by incomplete knowledge extraction and semantic misinterpretation, this paper introduces Layer-wise Relevance Propagation (LRP) to RAG hallucination detection for the first time, proposing an interpretable, fine-grained detection framework. Our method quantifies layer-wise relevance between inputs and outputs, constructs and resamples a relevance matrix, and integrates multiple classifiers—including XGBoost and MLP—for hallucination classification. Evaluated on multiple RAG hallucination benchmarks, our approach achieves a 12.3% improvement in F1-score over state-of-the-art methods. It exhibits strong generalizability across diverse RAG architectures and provides model-agnostic interpretability through LRP-based attribution. This work establishes a novel paradigm for assessing RAG reliability, bridging the gap between hallucination detection and transparent, explainable evaluation.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Retrieval-Augmented Generation (RAG) has become a primary technique for mitigating hallucinations in large language models (LLMs). However, incomplete knowledge extraction and insufficient understanding can still mislead LLMs to produce irrelevant or even contradictory responses, which means hallucinations persist in RAG. In this paper, we propose LRP4RAG, a method based on the Layer-wise Relevance Propagation (LRP) algorithm for detecting hallucinations in RAG. Specifically, we first utilize LRP to compute the relevance between the input and output of the RAG generator. We then apply further extraction and resampling to the relevance matrix. The processed relevance data are input into multiple classifiers to determine whether the output contains hallucinations. To the best of our knowledge, this is the first time that LRP has been used for detecting RAG hallucinations, and extensive experiments demonstrate that LRP4RAG outperforms existing baselines.

Problem

• Detecting hallucinations in Retrieval-Augmented Generation (RAG)
• Identifying irrelevant or contradictory responses in LLMs
• Improving accuracy of hallucination detection using LRP algorithm

Innovation

• Uses Layer-wise Relevance Propagation (LRP)
• Computes input-output relevance in RAG
• Applies classifiers to detect hallucinations