Hallucination Detection in LLMs via Topological Divergence on Attention Graphs

2025-04-14

AI Summary

Hallucination in large language model (LLM) generation severely hinders trustworthy deployment. To address this, we propose TOHA, the first hallucination detector that leverages topological discrepancies—quantified via persistent homology distance—between attention-induced subgraphs of prompts and responses. Specifically, TOHA constructs attention subgraphs per layer and head, then measures their topological divergence; we empirically identify that high divergence in certain attention heads strongly correlates with hallucinatory outputs. The method is inherently compatible with retrieval-augmented generation (RAG) pipelines, requiring no fine-tuning or external knowledge bases. Evaluated on multiple question-answering and data-to-text benchmarks, TOHA achieves state-of-the-art performance. We publicly release two high-quality, human-annotated hallucination datasets. Extensive experiments further demonstrate strong generalization across diverse LLMs, unseen datasets, and out-of-domain scenarios.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Hallucination, i.e., generating factually incorrect content, remains a critical challenge for large language models (LLMs). We introduce TOHA, a TOpology-based HAllucination detector in the RAG setting, which leverages a topological divergence metric to quantify the structural properties of graphs induced by attention matrices. Examining the topological divergence between prompt and response subgraphs reveals consistent patterns: higher divergence values in specific attention heads correlate with hallucinated outputs, independent of the dataset. Extensive experiments, including evaluation on question answering and data-to-text tasks, show that our approach achieves state-of-the-art or competitive results on several benchmarks, two of which were annotated by us and are being publicly released to facilitate further research. Beyond its strong in-domain performance, TOHA maintains remarkable domain transferability across multiple open-source LLMs. Our findings suggest that analyzing the topological structure of attention matrices can serve as an efficient and robust indicator of factual reliability in LLMs.

Problem

• Detecting hallucination in LLMs via attention graph topology
• Quantifying structural divergence in prompt-response attention graphs
• Evaluating factual reliability using topological analysis of attention

Innovation

• Topological divergence on attention graphs
• Quantify structural properties of attention matrices
• Domain transferability across multiple LLMs