LANCET: Neural Intervention via Structural Entropy for Mitigating Faithfulness Hallucinations in LLMs

2026-01-04

arXiv.org

AI Summary

This work addresses the challenge of hallucination in large language models, which arises from the distributed nature of neural information propagation and remains inadequately mitigated by existing intervention methods. The authors introduce structural entropy into neural pathway analysis for the first time, leveraging a hallucination discrepancy ratio and gradient-driven contrast to precisely identify and map forward-propagation paths associated with hallucination-prone neurons. Building on this insight, they propose a hierarchical, surgery-like neural intervention mechanism that transcends the limitations of conventional node-level or coarse-grained suppression strategies. Evaluated across multiple hallucination benchmarks, the method significantly outperforms state-of-the-art approaches, effectively curbing hallucinations while preserving the model’s general capabilities.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large Language Models have revolutionized information processing, yet their reliability is severely compromised by faithfulness hallucinations. While current approaches attempt to mitigate this issue through node-level adjustments or coarse suppression, they often overlook the distributed nature of neural information, leading to imprecise interventions. Recognizing that hallucinations propagate through specific forward transmission pathways like an infection, we aim to surgically block this flow using precise structural analysis. To leverage this, we propose Lancet, a novel framework that achieves precise neural intervention by leveraging structural entropy and hallucination difference ratios. Lancet first locates hallucination-prone neurons via gradient-driven contrastive analysis, then maps their propagation pathways by minimizing structural entropy, and finally implements a hierarchical intervention strategy that preserves general model capabilities. Comprehensive evaluations across hallucination benchmark datasets demonstrate that Lancet significantly outperforms state-of-the-art methods, validating the effectiveness of our surgical approach to neural intervention.

Problem

• faithfulness hallucinations
• neural intervention
• structural entropy
• Large Language Models
• hallucination propagation

Innovation

• structural entropy
• neural intervention
• faithfulness hallucinations
• gradient-driven contrastive analysis
• hierarchical intervention