PerturboLLaVA: Reducing Multimodal Hallucinations with Perturbative Visual Training

2025-03-09

AI Summary

This paper addresses the hallucination problem in multimodal large language models (MLLMs) for dense image captioning, which stems from excessive reliance on linguistic priors. To mitigate this, we propose a vision-enhancement method based on adversarial text perturbation. Our contributions are threefold: (1) We introduce HalFscore, a concept-level hallucination evaluation metric enabling fine-grained, quantitative hallucination measurement—the first of its kind. (2) We design a perturbed-text training paradigm guided by a language graph and jointly optimized via vision–language fine-tuning, explicitly attenuating linguistic priors while strengthening visual grounding. (3) The method requires intervention only during training, incurring no additional inference overhead. Experiments demonstrate significant hallucination reduction across dense captioning and general multimodal benchmarks, with consistent performance gains over state-of-the-art approaches.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

This paper aims to address the challenge of hallucinations in Multimodal Large Language Models (MLLMs) particularly for dense image captioning tasks. To tackle the challenge, we identify the current lack of a metric that finely measures the caption quality in concept level. We hereby introduce HalFscore, a novel metric built upon the language graph and is designed to evaluate both the accuracy and completeness of dense captions at a granular level. Additionally, we identify the root cause of hallucination as the model's over-reliance on its language prior. To address this, we propose PerturboLLaVA, which reduces the model's reliance on the language prior by incorporating adversarially perturbed text during training. This method enhances the model's focus on visual inputs, effectively reducing hallucinations and producing accurate, image-grounded descriptions without incurring additional computational overhead. PerturboLLaVA significantly improves the fidelity of generated captions, outperforming existing approaches in handling multimodal hallucinations and achieving improved performance across general multimodal benchmarks.

Problem

• Addresses hallucinations in Multimodal Large Language Models
• Introduces HalFscore for granular caption quality evaluation
• Reduces language prior reliance with PerturboLLaVA training

Innovation

• Introduces HalFscore for granular caption evaluation
• Proposes PerturboLLaVA to reduce language prior reliance
• Uses adversarially perturbed text to enhance visual focus