The Courtroom Trial of Pixels: Robust Image Manipulation Localization via Adversarial Evidence and Reinforcement Learning Judgment

2026-04-16

AI Summary

This work addresses the challenge that existing image manipulation localization methods struggle to explicitly model contrasting evidence between forged and authentic regions when tampering traces are subtle or obscured by post-processing, often yielding ambiguous and unreliable predictions. To overcome this limitation, the authors propose a forensic adversarial framework that formulates localization as a three-party game among an accusing stream, a defending stream, and a judge model. A multi-scale shared encoder generates opposing evidences, while a reinforcement learning–driven dynamic debate mechanism enables iterative re-reasoning over uncertain regions. The approach innovatively integrates bidirectional discrepancy suppression, cascaded multi-level fusion, and a judge model guided by advantage-based rewards and soft IoU, further refined through entropy and cross-hypothesis consistency calibration. Extensive experiments demonstrate significant performance gains over state-of-the-art methods, markedly enhancing robustness and reliability under complex interference.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

Although some existing image manipulation localization (IML) methods incorporate authenticity-related supervision, this information is typically utilized merely as an auxiliary training signal to enhance the model's sensitivity to manipulation artifacts, rather than being explicitly modeled as localization evidence opposing the manipulated regions. Consequently, when manipulation traces are subtle or degraded by post-processing and noise, these methods struggle to explicitly compare manipulated and authentic evidence, resulting in unreliable predictions in ambiguous areas. To address these issues, we propose a courtroom-style adjudication framework that regards IML task as the confrontation of evidence followed by judgment. The framework comprises a prosecution stream, a defense stream, and a judge model. We first build a dual-hypothesis segmentation architecture on a shared multi-scale encoder, in which the prosecution stream asserts manipulation and the defense stream asserts authenticity. Guided by edge priors, it produces evidence for manipulated and authentic regions through cascaded multi-level fusion, bidirectional disagreement suppression, and dynamic debate refinement. We further develop a reinforcement learning judge model that performs strategic re-inference and refinement on uncertain regions, yielding a manipulated-region mask. The judge model is trained with advantage-based rewards and a soft-IoU objective, and reliability is calibrated via entropy and cross-hypothesis consistency. Experimental results show that our model achieves superior average performance compared with SOTA IML methods.

Problem

• image manipulation localization
• evidence comparison
• ambiguous regions
• manipulation artifacts
• post-processing degradation

Innovation

• image manipulation localization
• adversarial evidence
• reinforcement learning
• dual-hypothesis segmentation
• courtroom-style adjudication