Knowing When Not to Answer: Evaluating Abstention in Multimodal Reasoning Systems

2026-04-16

AI Summary

Existing vision-language models and multi-agent systems lack effective refusal mechanisms when confronted with multimodal questions that lack sufficient evidence, often producing unreliable outputs. This work presents the first systematic investigation of refusal behavior in multimodal settings, introducing the MM-AQA benchmark, which constructs fine-grained unanswerable samples by manipulating visual dependency and evidential sufficiency, alongside a refusal-aware evaluation framework. Experiments reveal that models rarely refuse under standard prompting; while multi-agent systems increase refusal rates, they introduce a trade-off between accuracy and appropriate refusal. Furthermore, sequential architectures outperform iterative ones, indicating that robust refusal capability hinges on calibration rather than reasoning depth.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Social and technical mechanisms of refusal for web technologies and applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Effective abstention (EA), recognizing evidence insufficiency and refraining from answering, is critical for reliable multimodal systems. Yet existing evaluation paradigms for vision-language models (VLMs) and multi-agent systems (MAS) assume answerability, pushing models to always respond. Abstention has been studied in text-only settings but remains underexplored multimodally; current benchmarks either ignore unanswerability or rely on coarse methods that miss realistic failure modes. We introduce MM-AQA, a benchmark that constructs unanswerable instances from answerable ones via transformations along two axes: visual modality dependency and evidence sufficiency. Evaluating three frontier VLMs spanning closed and open-source models and two MAS architectures across 2079 samples, we find: (1) under standard prompting, VLMs rarely abstain; even simple confidence baselines outperform this setup, (2) MAS improves abstention but introduces an accuracy-abstention trade-off, (3) sequential designs match or exceed iterative variants, suggesting the bottleneck is miscalibration rather than reasoning depth, and (4) models abstain when image or text evidence is absent, but attempt reconciliation with degraded or contradictory evidence. Effective multimodal abstention requires abstention-aware training rather than better prompting or more agents.

Problem

• abstention
• multimodal reasoning
• vision-language models
• unanswerable questions
• evaluation benchmark

Innovation

• multimodal abstention
• vision-language models
• multi-agent systems
• evidence sufficiency
• MM-AQA