TDGNet: Hallucination Detection in Diffusion Language Models via Temporal Dynamic Graphs

2026-02-08

AI Summary

This work addresses the challenge of hallucination detection in diffusion-based language models (D-LLMs), where factual evidence is dispersed and dynamically evolves throughout the denoising process, rendering conventional autoregressive approaches ineffective. To this end, the authors propose TDGNet, the first method to incorporate temporal dynamic graphs for hallucination detection in D-LLMs. TDGNet constructs a dynamic graph structure via sparse token-level attention maps, updates token representations through graph message passing, and aggregates evidence across the entire denoising trajectory using temporal attention—enabling efficient single-pass inference. Evaluated on LLaDA-8B and Dream-7B models across multiple question-answering benchmarks, TDGNet significantly outperforms baseline methods operating at the output level, latent space level, and static graph level in terms of AUROC, while maintaining low inference overhead.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Diffusion language models (D-LLMs) offer parallel denoising and bidirectional context, but hallucination detection for D-LLMs remains underexplored. Prior detectors developed for auto-regressive LLMs typically rely on single-pass cues and do not directly transfer to diffusion generation, where factuality evidence is distributed across the denoising trajectory and may appear, drift, or be self-corrected over time. We introduce TDGNet, a temporal dynamic graph framework that formulates hallucination detection as learning over evolving token-level attention graphs. At each denoising step, we sparsify the attention graph and update per-token memories via message passing, then apply temporal attention to aggregate trajectory-wide evidence for final prediction. Experiments on LLaDA-8B and Dream-7B across QA benchmarks show consistent AUROC improvements over output-based, latent-based, and static-graph baselines, with single-pass inference and modest overhead. These results highlight the importance of temporal reasoning on attention graphs for robust hallucination detection in diffusion language models.

Problem

• hallucination detection
• diffusion language models
• temporal reasoning
• attention graphs
• denoising trajectory

Innovation

• Temporal Dynamic Graph
• Diffusion Language Models
• Hallucination Detection
• Message Passing
• Temporal Attention