Evidence Triangulation for Multimodal Fact-Checking in the Wild

2026-06-30

AI Summary

This work addresses the limitations of existing approaches that rely on synthetic data and artificial benchmarks, which struggle to handle the complex multimodal misinformation prevalent in real-world social media and fail to model fine-grained semantic relationships among images, text, and external evidence. To bridge this gap, the authors propose TRENT, a novel model that introduces evidence triangulation into real-world multimodal fact-checking for the first time. TRENT employs a three-way parallel cross-attention mechanism to fuse visual, textual, and retrieved evidential signals and incorporates a relation-aware fusion module to explicitly model entailment and contradiction logic. Additionally, the authors construct X-POSE, the first realistic multimodal benchmark derived from the social platform X, integrating full news articles with community annotations. Experiments demonstrate that TRENT significantly outperforms both specialized models and off-the-shelf vision-language models on X-POSE, exhibiting superior robustness and generalization.

Abstract

The proliferation of multimedia content on social platforms has fueled multimodal misinformation, where images are used to reinforce false claims. Consequently, Multimodal Fact-Checking (MFC) has emerged as an increasingly important research area. However, current progress is hindered by a reliance on synthetic training data and curated benchmarks that fail to capture the complexity of in-the-wild data. Furthermore, existing detection models rely on restricted intra-modality consistency or unconstrained all-to-all fusion, failing to capture nuanced relations between posts and external evidence. To address these limitations, we introduce X-POSE, a benchmark of real-world, community-annotated multimodal posts from X (formerly Twitter), augmented with full-length news articles retrieved via VLM-optimized search. Additionally, we propose TRENT, a novel MFC model that performs evidence triangulation using three parallel cross-attention streams alongside a relational fusion mechanism that explicitly models entailment and contradiction. Extensive evaluations demonstrate that TRENT consistently outperforms state-of-the-art specialized models and commercial VLMs. The code, prompt templates, and dataset are available at https://github.com/stevejpapad/evidence-triangulation

Problem

• Multimodal Fact-Checking
• evidence triangulation
• in-the-wild data
• misinformation
• cross-modal reasoning

Innovation

• Evidence Triangulation
• Multimodal Fact-Checking
• Cross-Attention Streams
• Relational Fusion
• In-the-Wild Benchmark