Mitigating Visual Hallucinations in Multimodal Systems through Retrieval-Augmented Reliability-Aware Inference

2026-06-14

AI Summary

This work addresses the tendency of multimodal large language models to exhibit overconfidence and visual hallucinations when faced with insufficient visual evidence or semantic conflicts, compounded by the absence of instance-level reliability assessment mechanisms. To enhance model calibration without requiring retraining, the authors propose a retrieval-augmented, reliability-aware reasoning framework. This approach leverages an external visual evidence bank, integrating pretrained visual embeddings, normalized nearest-neighbor retrieval, and multidimensional reliability metrics—including similarity strength, category consistency, evidence boundary, and entropy-based uncertainty—alongside a decision gating mechanism that dynamically chooses to accept, respond cautiously, or abstain from answering. Evaluated on ImageNet-100, the method improves the accuracy of accepted predictions from 85.84% to 88.88% at a coverage rate of 89.04%, while reducing hallucination error rates by 3.04 percentage points.

Abstract

Multimodal large language models (MLLMs) have demonstrated strong capabilities in vision-language understanding and natural-language response generation. However, these systems can still produce overconfident predictions and hallucination-like outputs, particularly when the visual evidence is weak, ambiguous, or semantically inconsistent. Most existing approaches focus on improving multimodal representation alignment or retrieval-augmented generation, while providing limited mechanisms to quantify instance-level prediction reliability or identify incorrect visual outputs. This work proposes a retrieval-augmented reliability-aware inference framework for trustworthy multimodal visual understanding. The proposed framework constructs an external visual evidence database using pretrained visual embeddings and nearest-neighbor retrieval over normalized feature representations. Retrieved evidence is used to estimate prediction trustworthiness through multiple reliability indicators, including similarity strength, class-support agreement, evidence margin, entropy-based uncertainty, and an aggregate reliability score. Based on these signals, a decision gate determines whether the system should accept the prediction, answer with caution, or abstain/fallback when evidence is insufficient. A multimodal response-generation layer then produces a final user-facing response conditioned on the reliability decision. Experiments on ImageNet-100 demonstrate that the proposed reliability-aware framework improves accepted prediction accuracy from 85.84\% to 88.88\% at 89.04\% coverage. The hallucination-like accepted wrong-answer rate is reduced from 14.16\% to 11.12\%. These results show that integrating retrieval evidence, reliability estimation, and selective decision gating can improve calibration and reduce overconfident visual errors without retraining large multimodal models.

Problem

• visual hallucinations
• multimodal systems
• reliability estimation
• prediction trustworthiness
• retrieval-augmented inference

Innovation

• reliability-aware inference
• retrieval-augmented generation
• visual hallucination mitigation
• multimodal trustworthiness
• selective decision gating