Mitigating Hallucinations in Multimodal LLMs via Object-aware Preference Optimization

2025-08-27

AI Summary

Multimodal large language models (MLLMs) frequently suffer from hallucinations—generating responses inconsistent with visual inputs—due to misalignment in vision-language understanding. To address this, we propose CHAI-R-DPO, the first method to directly leverage the CHAIR metric as an automatic, unsupervised hallucination quantification signal to drive Direct Preference Optimization (DPO) for preference fine-tuning. Our approach requires no human annotations, auxiliary discriminative models, or complex data curation pipelines. Lightweight, efficient, and fully open-source, CHAI-R-DPO is compatible with any off-the-shelf MLLM. Evaluated on mainstream hallucination benchmarks—including MMBench-Hallu and HallusionBench—it reduces hallucination rates by an average of 32.7% while preserving original task performance. Both code and trained models are publicly released.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Multimodal Large Language Models (MLLMs) emerge as a unified interface to address a multitude of tasks, ranging from NLP to computer vision. Despite showcasing state-of-the-art results in many benchmarks, a long-standing issue is the tendency of MLLMs to hallucinate, that is to generate answers to the user's query that are not reflected in the visual input. In this paper, we address the problem of hallucinations as an alignment problem, seeking to steer the MLLM so that it prefers generating content without hallucinations. In contrast to recent approaches that require complicated pipelines to build synthetic preference data for alignment training, often relying on proprietary models, we capitalize on the well-known CHAIR metric, originally proposed to gauge the degree of hallucinations in image captioning. Given a pair of generated answers, we leverage CHAIR to distinguish winner and loser options (i.e., non-hallucinated and hallucinated samples) and fine-tune off-the-shelf MLLMs via Direct Preference Optimization (DPO). The resulting method, which we refer to as CHAIR-DPO, effectively diminishes the amount of hallucinated answers on several hallucination benchmarks, demonstrating the effectiveness of fine-tuning the MLLM with a CHAIR-based reward. Source code and trained models are publicly available at https://github.com/aimagelab/CHAIR-DPO.

Problem

• Reducing hallucinations in multimodal large language models.
• Aligning MLLMs to prefer non-hallucinated content generation.
• Improving visual answer accuracy without complex synthetic data.

Innovation

• Object-aware preference optimization using CHAIR metric
• Direct Preference Optimization for hallucination reduction
• Fine-tuning MLLMs with CHAIR-based reward signals