GAP-SAM: A Global Artifact Prior for Generalizable AI-Generated Image Manipulation Localization

2026-08-21

AI Summary

为了解决AI生成图像编辑定位的泛化问题，提出GAP-SAM方法，通过全局伪影令牌和特征金字塔调整来提高定位准确性。

Abstract

AI-generated image manipulation localization identifies edited pixels, but its OOD performance lags behind image-level detection partly because pixel supervision entangles forensic evidence with dataset-specific mask geometry and semantic boundaries. Extending image-level distribution alignment to localization, we construct COCO-ControlNet with source-image Canny edges and depth maps to align semantics and geometry, improving OOD performance across multiple localizers. Yet tighter Mask-VAE Reconstruction Alignment (Mask-VAE) underperforms COCO-ControlNet, showing that VAE reconstruction artifacts transfer poorly to local diffusion-inpainting artifacts. We also identify \emph{boundary adhesion}, where fine-tuned segmentation models snap predictions to semantic object contours rather than true manipulation boundaries. These findings motivate GAP-SAM, which encodes an image and its frozen VAE reconstruction into a global artifact token and injects it into SAM3's feature pyramid via zero-gated FiLM before pixel decoding. Without prescribing a spatial region, this token modulates dense decoding to preserve localization while suppressing semantic-boundary shortcuts. Across six datasets, GAP-SAM averages 79.8 Pixel-F1, outperforming the strongest prior method by 12.6 points. It also performs best at every tested severity of JPEG compression, Gaussian blur, and resizing.

Problem

• AI-generated image manipulation
• localization
• out-of-distribution performance
• pixel supervision
• semantic boundaries

Innovation

• GAP-SAM
• Global Artifact Prior
• Zero-gated FiLM
• Semantic-boundary Shortcuts Suppression
• Generalizable AI-Generated Image Manipulation Localization