Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation

2026-06-29

AI Summary

This work addresses the tendency of multimodal large language models to generate hallucinations by over-relying on linguistic priors while neglecting visual evidence. To mitigate this issue, the authors propose a preference optimization method oriented toward visual evidence strength: by comparing the same faithful response under varying levels of visual support, the model is encouraged to place greater trust in the visual content it already attends to. The key innovation lies in extending preference learning from response quality to ranking based on visual evidence strength, complemented by attention-guided evidence enhancement, Ordered Preference Optimization based on evidence strength (OPPO), and span- or token-level regularization to improve training stability and local visual sensitivity. Experiments demonstrate that the proposed approach significantly outperforms existing methods across multiple hallucination and general multimodal benchmarks, effectively alleviating hallucination.

Abstract

Multimodal Large Language Models (MLLMs) are prone to hallucination as their generation preferences are insufficiently calibrated to visual evidence, causing them to fall back on linguistic priors, rather than faithful grounding. In this work, we start from an empirical observation: when query-relevant visual evidence is explicitly strengthened using the model's own attention, generation becomes more accurate, suggesting that many failures do not arise solely from missing perception, but from an insufficient tendency to trust the evidence the model has already attended to. Motivated by this finding, we propose Oriented Pickup Preference Optimization (\texttt{OPPO}), an evidence-aware alignment objective that learns preferences over the strength of visual evidence, rather than only response quality. Concretely, \texttt{OPPO} contrasts the same faithful response under stronger, anchored, weaker-evidence views, turning naive visual preference into ordered visual-evidence alignment. We further combine this objective with fine-grained span-level and token-level regularization to stabilize the training. Besides, we provide a theoretical analysis showing that ordered evidence margins induce a positive lower bound on local visual sensitivity. Extensive evaluations across hallucination and general-purpose benchmarks demonstrate that \texttt{OPPO} consistently outperforms baseline methods.

Problem

• multimodal hallucination
• visual grounding
• preference optimization
• evidence calibration
• MLLMs

Innovation

• multimodal hallucination
• preference optimization
• visual evidence alignment
• attention calibration
• evidence-aware learning