Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models

2024-10-04

arXiv.org

AI Summary

Multimodal large language models (MLLMs) suffer from hallucinations—generating text inconsistent with image content—due to the text decoder’s sensitivity to visual tokens. To address this, we propose a “Look Again” decoding paradigm: when inference uncertainty is high, visual tokens are dynamically modeled as key-value memory and injected without latency into the feed-forward network (FFN) intermediate layers, enabling trigger-based visual-text factual alignment. This method requires no fine-tuning, is fully plug-and-play, and introduces, for the first time, a cognition-inspired visual re-attending mechanism directly into the decoding process. Evaluated on benchmarks including MMBench and OCRBench, it significantly reduces hallucination rates while preserving or improving original task performance—all at zero additional inference overhead.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Despite their impressive capabilities, multimodal large language models (MLLMs) are prone to hallucinations, i.e., the generated content that is nonsensical or unfaithful to input sources. Unlike in LLMs, hallucinations in MLLMs often stem from the sensitivity of text decoder to visual tokens, leading to a phenomenon akin to"amnesia"about visual information. To address this issue, we propose MemVR, a novel decoding paradigm inspired by common cognition: when the memory of an image seen the moment before is forgotten, people will look at it again for factual answers. Following this principle, we treat visual tokens as supplementary evidence, re-injecting them into the MLLM through Feed Forward Network (FFN) as"key-value memory"at the middle trigger layer. This"look-twice"mechanism occurs when the model exhibits high uncertainty during inference, effectively enhancing factual alignment. Comprehensive experimental evaluations demonstrate that MemVR significantly mitigates hallucination across various MLLMs and excels in general benchmarks without incurring additional time overhead. The implementation is available from https://github.com/1zhou-Wang/MemVR

Problem

• MLLMs generate unfaithful content due to visual token sensitivity
• Visual information is forgotten, causing hallucination in MLLMs
• Proposed solution reinjects visual tokens to enhance factual alignment

Innovation

• Re-injects visual tokens as key-value memory
• Uses FFN for mid-layer visual token integration
• Triggers look-twice mechanism during high uncertainty