Weakly-supervised Localization of Manipulated Image Regions Using Multi-resolution Learned Features

2025-05-29

AI Summary

To address the challenge of localizing tampered regions in real-world scenarios where pixel-level annotations are unavailable, this paper proposes a weakly supervised forgery localization framework. Methodologically, it fuses multi-view activation maps from an image-level detection network (WCBnet) with region priors derived from pre-trained segmentation models (e.g., Segment Anything, DeepLab, PSPNet), and refines the localization posterior distribution via Bayesian inference. Its key innovation lies in being the first to integrate multi-resolution feature learning with segmentation priors for weakly supervised localization—achieving high-precision, interpretable, and pixel-level localization without any pixel-level supervision. Evaluated on multiple benchmark datasets, the framework achieves state-of-the-art performance, significantly outperforming existing methods in localization accuracy. This demonstrates both the feasibility and practicality of reliable forgery region localization under purely image-level supervision.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: Localization, Mapping, and Navigation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Measurement, analysis, and circumvention of Web censorship

Abstract

The explosive growth of digital images and the widespread availability of image editing tools have made image manipulation detection an increasingly critical challenge. Current deep learning-based manipulation detection methods excel in achieving high image-level classification accuracy, they often fall short in terms of interpretability and localization of manipulated regions. Additionally, the absence of pixel-wise annotations in real-world scenarios limits the existing fully-supervised manipulation localization techniques. To address these challenges, we propose a novel weakly-supervised approach that integrates activation maps generated by image-level manipulation detection networks with segmentation maps from pre-trained models. Specifically, we build on our previous image-level work named WCBnet to produce multi-view feature maps which are subsequently fused for coarse localization. These coarse maps are then refined using detailed segmented regional information provided by pre-trained segmentation models (such as DeepLab, SegmentAnything and PSPnet), with Bayesian inference employed to enhance the manipulation localization. Experimental results demonstrate the effectiveness of our approach, highlighting the feasibility to localize image manipulations without relying on pixel-level labels.

Problem

• Detect manipulated image regions with weak supervision
• Improve interpretability and localization of image manipulations
• Localize manipulations without pixel-level annotations

Innovation

• Weakly-supervised localization using multi-resolution features
• Fusion of activation maps with pre-trained segmentation models
• Bayesian inference enhances manipulated region localization