THRONE: An Object-Based Hallucination Benchmark for the Free-Form Generations of Large Vision-Language Models

2024-05-08

Computer Vision and Pattern Recognition

AI Summary

Large vision-language models (LVLMs) suffer from pervasive object-level hallucinations (Type I) during open-ended generation, yet existing benchmarks predominantly evaluate factuality errors in closed-ended QA (Type II). These two hallucination types often exhibit inverse correlation, leading to severely inaccurate evaluation. Method: We propose the first automated benchmark targeting Type I hallucinations: (1) formally defining and quantifying Type I hallucinations; (2) building an end-to-end detection framework leveraging public LMs—without external APIs; (3) designing object-level prompting, multi-stage LM-assisted identification, and public-data-driven evaluation; and (4) introducing targeted data augmentation. Contributions/Results: Experiments reveal that state-of-the-art LVLMs show no reduction in Type I hallucinations despite improvements on existing metrics, exposing benchmark incompleteness. Our method simultaneously mitigates both Type I and Type II hallucinations, demonstrating effectiveness and generalizability across diverse LVLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Mitigating hallucinations in large vision-language models (LVLMs) remains an open problem. Recent benchmarks do not address hallucinations in open-ended free-form responses, which we term “Type I hallucinations”. Instead, they focus on hallucinations responding to very specific question formats-typically a multiple-choice response regarding a particular object or attribute-which we term “Type II hallucinations”. Additionally, such benchmarks often require external API calls to models which are subject to change. In practice, we observe that a reduction in Type II hallucinations does not lead to a reduction in Type I hallucinations but rather that the two forms of halluci-nations are often anti-correlated. To address this, we propose THRONE, a novel object-based automatic framework for quantitatively evaluating Type I hallucinations in LVLM free-form outputs. We use public language models (LMs) to identify hallucinations in LVLM responses and compute informative metrics. By evaluating a large selection of recent LVLMs using public datasets, we show that an improvement in existing metrics do not lead to a reduction in Type I hallucinations, and that established benchmarks for measuring Type I hallucinations are incomplete. Finally, we provide a simple and effective data augmentation method to reduce Type I and Type II hallucinations as a strong baseline.

Problem

• Mitigating Type I hallucinations in LVLM free-form responses
• Evaluating object-based hallucinations quantitatively in LVLMs
• Reducing Type I and Type II hallucinations via data augmentation

Innovation

• Object-based automatic framework for evaluating hallucinations
• Public LMs identify hallucinations in LVLM responses
• Data augmentation method reduces both hallucination types