Look Again Before You Abstain:Budgeted Conformal Evidence Acquisition for Reliable Vision-Language Model

2026-06-15

AI Summary

This work addresses the challenge of hallucination in large vision-language models (LVLMs), which often generate details absent from input images. While existing conformal abstention methods can control hallucination rates, they suffer from excessively high abstention rates, thereby discarding valuable visual evidence. To overcome this limitation, the authors propose Budget-Constrained Conformal Evidence Acquisition (BCEA), a novel framework that integrates active evidence acquisition into conformal prediction. BCEA introduces a third option—beyond answering or abstaining—allowing the model to perform structured image interventions such as cropping or resizing within a computational budget to reacquire evidence. Combining query-type-specific intervention strategies, post-acquisition score recalibration, and mechanisms preserving exchangeability, BCEA effectively controls hallucination rates at target levels across four open-source VLMs on POPE and COCO existence and spatial relation tasks, while significantly improving coverage and consistently outperforming abstention-only baselines.

Abstract

Large vision-language models (LVLMs) hallucinate: they assert visual details that the image does not support. A principled remedy is selective prediction with a distribution-free guarantee-verify each claim and abstain when the claim is not grounded, so that the hallucination rate among asserted claims is provably bounded. We show, however, that this guarantee is bought at a brutal price: to keep the hallucination rate below $5\%$ on a balanced object-existence benchmark, a state-of-the-art conformal filter must abstain on more than $80\%$ of claims. We argue that abstention is wasteful when more visual evidence is cheaply available, and introduce Budgeted Conformal Evidence Acquisition (BCEA), which replaces the binary answer/abstain decision with a three-way choice: answer, abstain, or acquire additional visual evidence by re-examining the image (zooming, cropping, or applying a claim-specific intervention) under a bounded compute budget. We make two observations. First, acquisition that is plugged naively into a calibrated filter breaks the statistical guarantee -- realized risk overshoots the target by up to $17$ points -- because the acquisition step destroys the exchangeability that conformal calibration relies on. Second, folding the entire acquisition policy into the score function and re-calibrating on post-acquisition scores \emph{restores} the finite-sample guarantee while still recovering coverage. BCEA further uses structured, claim-type-specific interventions. Across the POPE benchmark and COCO-constructed existence and spatial-relation claims, on four open VLMs, BCEA controls the hallucination rate at the target level and consistently improves coverage over a guaranteed-abstention baseline.

Problem

• hallucination
• selective prediction
• conformal prediction
• vision-language models
• abstention

Innovation

• Conformal Prediction
• Vision-Language Models
• Selective Prediction
• Evidence Acquisition
• Hallucination Control