APCD: Adaptive Path-Contrastive Decoding for Reliable Large Language Model Generation

2026-05-10

AI Summary

This work addresses the susceptibility of large language models to error propagation and hallucination during autoregressive decoding, which often stems from early suboptimal token choices. To mitigate this, the authors propose an adaptive path contrastive decoding framework that dynamically triggers multi-path exploration based on Shannon entropy and modulates the strength of interaction among candidate paths through a distributional divergence-aware mechanism. This approach enables efficient and reliable multi-path reasoning by jointly leveraging predictive uncertainty and contrastive learning, achieving timing-adaptive path expansion with tunable influence. Evaluated across eight benchmark datasets, the method significantly improves factual accuracy in generated text while maintaining competitive decoding efficiency.

Abstract

Large language models (LLMs) often suffer from hallucinations due to error accumulation in autoregressive decoding, where suboptimal early token choices misguide subsequent generation. Although multi-path decoding can improve robustness by exploring alternative trajectories, existing methods lack principled strategies for determining when to branch and how to regulate inter-path interactions. We propose Adaptive Path-Contrastive Decoding (APCD), a multi-path decoding framework that improves output reliability through adaptive exploration and controlled path interaction. APCD consists of two components: (1) Entropy-Driven Path Expansion, which delays branching until predictive uncertainty - measured by Shannon entropy over top candidate tokens - indicates multiple plausible continuations; and (2) Divergence-Aware Path Contrast, which encourages diverse reasoning trajectories while dynamically attenuating inter-path influence as prediction distributions diverge. Experiments on eight benchmarks demonstrate improved factual accuracy while maintaining decoding efficiency. Our code is available at https://github.com/zty-king/APCD.

Problem

• hallucination
• autoregressive decoding
• multi-path decoding
• error accumulation
• path interaction

Innovation

• Adaptive Path-Contrastive Decoding
• Entropy-Driven Path Expansion
• Divergence-Aware Path Contrast
• Multi-path Decoding
• Hallucination Mitigation