T-IMPACT: A Severity-Aware Benchmark for Contextual Image-Text Manipulation

2026-06-21

AI Summary

Existing image-text manipulation datasets struggle to quantify the fine-grained impact of multimodal manipulations on audience comprehension. To address this limitation, this work proposes T-IMPACT—a multimodal manipulation benchmark tailored for news scenarios—comprising 98,786 samples generated through semantic anchor extraction, spatial localization, localized image editing, and constrained text rewriting. Each sample is annotated with both continuous severity scores and coarse-grained severity labels. T-IMPACT introduces, for the first time, calibrated continuous severity signals and grounded metadata, enabling a shift from binary authenticity judgments to contextual assessments of manipulation impact. Experimental results demonstrate that while current models effectively discern authenticity, they exhibit weak alignment with human judgments in predicting the severity of manipulations.

Abstract

Recent advances in vision-language models and generative editing systems have made it increasingly easy to produce persuasive multimodal misinformation by altering images, text, or both jointly. However, existing datasets focus mainly on authenticity, out-of-context mismatch, or manipulation type, and rarely capture how strongly an edit changes the likely interpretation of a post. We introduce T-IMPACT, a first-release severity-aware benchmark for manipulated news-style image-text pairs. T-IMPACT contains 98,786 examples spanning pristine, image-only, text-only, and joint manipulations, with a calibrated continuous severity signal, coarse low/medium/high labels, and supporting grounding metadata. Starting from a news image-text pair, the pipeline extracts semantic anchors, grounds them spatially, performs localized image edits and constrained caption rewrites, and calibrates contextual-impact scores using limited human ratings. In this release, the calibrated continuous score is the primary severity target, while the low/medium/high bands should be interpreted as coarse operating buckets rather than balanced classes. Experiments show that current models recover some authenticity signal, but severity prediction remains substantially harder and only weakly aligned with human judgment. T-IMPACT provides an initial benchmark for studying multimodal manipulation beyond binary real/fake classification toward graded contextual impact.

Problem

• multimodal manipulation
• severity assessment
• contextual impact
• image-text misinformation
• benchmark

Innovation

• severity-aware benchmark
• contextual image-text manipulation
• multimodal misinformation
• calibrated impact scoring
• semantic anchoring