DecoPrompt : Decoding Prompts Reduces Hallucinations when Large Language Models Meet False Premises

2024-11-12

arXiv.org

AI Summary

Large language models (LLMs) frequently generate factual hallucinations when conditioned on false premises, undermining reliability in real-world applications. Method: This paper proposes a lightweight, transferable “decoding-time prompting” intervention that operates without access to internal logits—leveraging prompt entropy to quantify hallucination risk induced by false premises, enabling zero-shot prompt disentanglement and hallucination-free decoding, and integrating LLM self-feedback for prompt rewriting coupled with entropy-driven false-premise identification. Contribution/Results: Evaluated on two authoritative benchmarks, the method significantly reduces hallucination rates across diverse mainstream LLMs—including proprietary and ultra-large-scale models—demonstrating strong cross-model generalization. It establishes a novel paradigm for trustworthy reasoning in black-box settings, offering an efficient, plug-and-play solution for mitigating premise-induced hallucinations without model modification or fine-tuning.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

While large language models (LLMs) have demonstrated increasing power, they have also called upon studies on their hallucinated outputs that deviate from factually correct statements. In this paper, we focus on one important scenario of false premises, where LLMs are distracted by misaligned claims although the model possesses the required factual knowledge to answer original questions accurately. Inspired by the observation that entropy of the false-premise prompt is closely related to its likelihood to elicit hallucination generation, we propose a new prompting algorithm, named DecoPrompt, to mitigate hallucination. DecoPrompt leverages LLMs to"decode"the false-premise prompts without really eliciting hallucination output from LLMs. We perform experiments on two datasets, demonstrating that DecoPrompt can reduce hallucinations effectively on outputs from different LLMs. Moreover, DecoPrompt exhibits cross-model transferability, which facilitates its applications to scenarios such as LLMs of large sizes or unavailable model logits.

Problem

• Large Language Models
• Error Processing
• Factual Consistency

Innovation

• DecoPrompt
• Error Reduction
• Misinformation Mitigation