Is a Picture Worth a Thousand Words? Adaptive Multimodal Fact-Checking with Visual Evidence Necessity

2026-04-06

AI Summary

Existing multimodal fact-checking approaches often indiscriminately fuse visual evidence, inadvertently introducing noise and degrading accuracy. To address this limitation, this work proposes the AMuFC framework, which introduces—for the first time—a mechanism to assess the necessity of visual evidence. Specifically, an Analyzer agent evaluates whether visual information is beneficial for claim verification, and a Verifier agent adaptively integrates this judgment with retrieved textual evidence to make a final decision. This dual-agent collaborative architecture enables condition-driven multimodal verification, significantly outperforming prior methods across three benchmark datasets. Additionally, the authors release WebFC, a new dataset designed to support fact-checking evaluation under more realistic scenarios.

Abstract

Automated fact-checking is a crucial task not only in journalism but also across web platforms, where it supports a responsible information ecosystem and mitigates the harms of misinformation. While recent research has progressed from text-only to multimodal fact-checking, a prevailing assumption is that incorporating visual evidence universally improves performance. In this work, we challenge this assumption and show that indiscriminate use of multimodal evidence can reduce accuracy. To address this challenge, we propose AMuFC, a multimodal fact-checking framework that employs two collaborative agents with distinct roles for the adaptive use of visual evidence: An Analyzer determines whether visual evidence is necessary for claim verification, and a Verifier predicts claim veracity conditioned on both the retrieved evidence and the Analyzer's assessment. Experimental results on three datasets show that incorporating the Analyzer's assessment of visual evidence necessity into the Verifier's prediction yields substantial improvements in verification performance. In addition to all code, we release WebFC, a newly constructed dataset for evaluating fact-checking modules in a more realistic scenario, available at https://github.com/ssu-humane/AMuFC.

Problem

• multimodal fact-checking
• visual evidence necessity
• automated fact-checking
• misinformation mitigation

Innovation

• adaptive multimodal fact-checking
• visual evidence necessity
• collaborative agents
• claim verification
• AMuFC