Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling

2025-04-17

AI Summary

Vision-language models (VLMs) frequently exhibit visual hallucinations—e.g., fabricating nonexistent objects or actions—posing significant risks in safety-critical applications. Existing mitigation strategies fall into two categories: generation-time adjustment and post-hoc verification; however, the former relies on heuristic designs lacking correction capability, while the latter suffers from high rejection rates and inability to rectify erroneous outputs. This paper introduces REVERSE, the first framework unifying hallucination-aware fine-tuning with inference-time self-verification. We construct a 1.3M-sample hallucination verification dataset, design a lightweight self-supervised verification head, and integrate backtracking resampling decoding to enable real-time hallucination detection and dynamic output correction. On CHAIR-MSCOCO and HaloQuest, REVERSE reduces hallucination rates by 12% and 28%, respectively, achieving state-of-the-art performance. Crucially, it is the first method to enable *verifiable and correctable* hallucination suppression *during generation*.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Vision-Language Models (VLMs) excel at visual understanding but often suffer from visual hallucinations, where they generate descriptions of nonexistent objects, actions, or concepts, posing significant risks in safety-critical applications. Existing hallucination mitigation methods typically follow one of two paradigms: generation adjustment, which modifies decoding behavior to align text with visual inputs, and post-hoc verification, where external models assess and correct outputs. While effective, generation adjustment methods often rely on heuristics and lack correction mechanisms, while post-hoc verification is complicated, typically requiring multiple models and tending to reject outputs rather than refine them. In this work, we introduce REVERSE, a unified framework that integrates hallucination-aware training with on-the-fly self-verification. By leveraging a new hallucination-verification dataset containing over 1.3M semi-synthetic samples, along with a novel inference-time retrospective resampling technique, our approach enables VLMs to both detect hallucinations during generation and dynamically revise those hallucinations. Our evaluations show that REVERSE achieves state-of-the-art hallucination reduction, outperforming the best existing methods by up to 12% on CHAIR-MSCOCO and 28% on HaloQuest. Our dataset, model, and code are available at: https://reverse-vlm.github.io.

Problem

• Reducing visual hallucinations in Vision-Language Models
• Integrating hallucination-aware training with self-verification
• Improving accuracy in safety-critical applications

Innovation

• Unified framework integrating hallucination-aware training
• On-the-fly self-verification during generation
• Retrospective resampling for dynamic hallucination revision