Look Before You Leap: Factual Decoding with Internal Attribution Signals

2026-09-14

AI Summary

为解决大语言模型中的事实错误累积问题，提出DescaPE框架，通过利用内部信号控制解码过程，减少事实错误的雪球效应。

Abstract

Hallucination remains a critical challenge in large language models (LLMs), where early factual errors compound through autoregressive generation in a snowballing effect that neither post-hoc correction nor weight-level intervention can effectively preempt. We propose DescaPE (DEcoding Signal Control Against Path Error-snowballing), a decoding framework that leverages internal model signals to suppress hallucination-prone trajectories at inference time. Through sliding-window MLP ablation, we identify a factual-salient layer span within LLMs whose derived signal is selectively elevated for factual tokens and exhibits anomalous spikes at hallucination-prone steps. We train a lightweight probe to approximate this signal from a single forward pass and integrate it into candidate scoring to penalize high-risk continuations while rewarding factually grounded ones. Experiments across five factuality benchmarks on three LLMs demonstrate that DescaPE achieves factuality improvements over decoding-time baselines in multiple settings, while incurring only 1.10x latency overhead in our efficiency evaluation. Our code is available at https://github.com/hayeonggg/DESCAPE.

Problem

• Hallucination
• Large Language Models
• Autoregressive Generation
• Factual Errors

Innovation

• DescaPE
• internal model signals
• factual-salient layer span
• lightweight probe
• candidate scoring