PAS : Prelim Attention Score for Detecting Object Hallucinations in Large Vision--Language Models

2025-11-14

AI Summary

Large Vision-Language Models (LVLMs) suffer from object hallucination—generating outputs inconsistent with image content due to excessive reliance on preceding textual tokens rather than visual input. This work identifies anomalous dominance of prompt tokens in the decoder’s self-attention mechanism as a core cause. To address this, we propose PAS (Prompt-Attention Sensitivity), a training-free, lightweight, and real-time hallucination detection metric. PAS quantifies prompt-token dependency via decoder self-attention weights and jointly evaluates image–prediction association using conditional mutual information. Evaluated across major LVLMs (e.g., LLaVA, Qwen-VL) and standard benchmarks (POPE, HalluBench), PAS achieves state-of-the-art detection performance. Crucially, it enables online filtering and intervention during inference—without modifying model architecture or fine-tuning—thereby establishing a new paradigm for trustworthy LVLM deployment.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large vision-language models (LVLMs) are powerful, yet they remain unreliable due to object hallucinations. In this work, we show that in many hallucinatory predictions the LVLM effectively ignores the image and instead relies on previously generated output (prelim) tokens to infer new objects. We quantify this behavior via the mutual information between the image and the predicted object conditioned on the prelim, demonstrating that weak image dependence strongly correlates with hallucination. Building on this finding, we introduce the Prelim Attention Score (PAS), a lightweight, training-free signal computed from attention weights over prelim tokens. PAS requires no additional forward passes and can be computed on the fly during inference. Exploiting this previously overlooked signal, PAS achieves state-of-the-art object-hallucination detection across multiple models and datasets, enabling real-time filtering and intervention.

Problem

• Detecting object hallucinations in large vision-language models
• Quantifying weak image dependence causing false object predictions
• Providing real-time hallucination detection without additional training

Innovation

• PAS uses attention weights over prelim tokens
• It detects hallucinations via weak image dependence
• This training-free method enables real-time intervention