Towards Generalized Image Manipulation Localization via Score-based Model

2026-05-16

AI Summary

This work addresses the limited generalization of existing image manipulation localization methods when confronted with unseen tampering types. To overcome this challenge, the authors propose DiffIML, a novel framework that introduces score-based generative modeling to this task for the first time. By approximating the gradient of the log-likelihood of mask distributions and incorporating structural priors, DiffIML iteratively recovers coherent masks from noise, thereby circumventing the fragility inherent in discriminative models. The approach employs a lightweight mask-specific VAE and a decoupled denoising UNet, further enhanced by edge-aware supervision and error priors. Extensive experiments demonstrate that DiffIML significantly outperforms state-of-the-art methods across eight non-generative and three generative benchmarks, exhibiting exceptional generalization to both unseen tampering types and datasets.

Abstract

With the rapid evolution of synthetic media, Image Manipulation Localization (IML) has emerged as a critical component in multimedia forensics for ensuring the integrity of digital content. However, generalization remains a core challenge, as existing discriminative methods typically learn a fixed decision boundary that tends to overfit to specific training artifacts and fails to adapt to unseen manipulation types. To address this, we propose DiffIML, a novel framework that introduces score-based generative modeling to IML. Diverging from the direct estimation of hard boundaries, DiffIML approximates the score function, the gradient of the log-likelihood, to capture the intrinsic geometric topology of mask distributions. This paradigm leverages structural priors to iteratively recover coherent masks from noise, thereby circumventing the brittleness associated with discriminative models. Under this formulation, diffusion models serve as an effective numerical solver for the learned score function.To ensure practicality, we respectively resolve the efficiency and stability bottlenecks of standard diffusion by: (1) utilizing a Lightweight Mask-Specific VAE for fast latent-space process and a decoupled architecture with a lightweight denoising UNet, (2) edge supervision and error prior to mitigate error accumulation during sampling. Extensive experiments of two distinct protocols on eight non-generative and three generative benchmarks demonstrate that DiffIML consistently outperforms state-of-the-art methods, yielding remarkable generalization improvements on diverse unseen datasets. The code will be publicly available.

Problem

• Image Manipulation Localization
• generalization
• synthetic media
• multimedia forensics
• unseen manipulation types

Innovation

• score-based generative modeling
• image manipulation localization
• diffusion models
• generalization
• mask recovery