PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset

2024-03-17

AI Summary

To address the prevalent visual hallucination issue in multimodal large language models (MLLMs) during visual question answering (VQA), this paper introduces VHE—the first large-scale, structured visual hallucination evaluation framework. Methodologically, we construct PhD, a novel benchmark dataset covering five core visual recognition tasks and four question-asking patterns—including inaccurate context and counterfactual images—establishing the first “task × pattern” two-dimensional evaluation paradigm. We pioneer the systematic integration of AI-generated counterfactual images and context perturbation mechanisms, and design a ChatGPT-assisted semi-automated hallucination data curation pipeline. The dataset comprises 14K real-world images, 750 counterfactual images, and 102K VQA triplets. Empirical analysis reveals strong task- and pattern-dependent hallucination behaviors across MLLMs, providing both a standardized, reproducible benchmark and a new methodological paradigm for objective visual hallucination assessment.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Multimodal Large Language Models (MLLMs) hallucinate, resulting in an emerging topic of visual hallucination evaluation (VHE). This paper contributes a ChatGPT-Prompted visual hallucination evaluation Dataset (PhD) for objective VHE at a large scale. The essence of VHE is to ask an MLLM questions about specific images to assess its susceptibility to hallucination. Depending on what to ask (objects, attributes, sentiment, etc.) and how the questions are asked, we structure PhD along two dimensions, i.e., task and mode. Five visual recognition tasks, ranging from low-level (object / attribute recognition) to middle-level (sentiment / position recognition and counting), are considered. Besides a normal visual QA mode, which we term PhD-base, PhD also asks questions with inaccurate context (PhD-iac) or with incorrect context (PhD-icc), or with AI-generated counter common sense images (PhD-ccs). We construct PhD by a ChatGPT-assisted semi-automated pipeline, encompassing four pivotal modules: task-specific hallucinatory item (hitem) selection, hitem-embedded question generation, inaccurate / incorrect context generation, and counter-common-sense (CCS) image generation. With over 14k daily images, 750 CCS images and 102k VQA triplets in total, PhD reveals considerable variability in MLLMs' performance across various modes and tasks, offering valuable insights into the nature of hallucination. As such, PhD stands as a potent tool not only for VHE but may also play a significant role in the refinement of MLLMs.

Problem

• Evaluates visual hallucination in Multimodal Large Language Models (MLLMs).
• Assesses MLLM susceptibility to hallucination across diverse tasks and modes.
• Provides a large-scale dataset for objective visual hallucination evaluation (VHE).

Innovation

• ChatGPT-assisted semi-automated pipeline
• Task-specific hallucinatory item selection
• Counter-common-sense image generation