Bridging the Micro--Macro Gap: Frequency-Aware Semantic Alignment for Image Manipulation Localization

2026-04-14

AI Summary

Existing image manipulation localization methods struggle to simultaneously handle both conventional forgeries and localized, photorealistic edits generated by diffusion models, primarily due to the disconnect between fine-grained forensic cues and high-level semantic understanding. To address this, this work proposes FASA, a unified framework that jointly models frequency-domain artifacts and semantic consistency for the first time. FASA employs an adaptive dual-band DCT to extract frequency-sensitive features, leverages block-wise contrastive learning on a frozen CLIP model to capture semantic priors, and introduces a semantic-frequency side adapter to enable multi-scale interaction. A prototype-guided, frequency-gated mask decoder then predicts tampered regions. The method achieves state-of-the-art performance on OpenSDI and multiple traditional benchmarks, demonstrating exceptional generalization across generators and datasets, as well as robustness to image degradation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Data transparency and provenance

Abstract

As generative image editing advances, image manipulation localization (IML) must handle both traditional manipulations with conspicuous forensic artifacts and diffusion-generated edits that appear locally realistic. Existing methods typically rely on either low-level forensic cues or high-level semantics alone, leading to a fundamental micro--macro gap. To bridge this gap, we propose FASA, a unified framework for localizing both traditional and diffusion-generated manipulations. Specifically, we extract manipulation-sensitive frequency cues through an adaptive dual-band DCT module and learn manipulation-aware semantic priors via patch-level contrastive alignment on frozen CLIP representations. We then inject these priors into a hierarchical frequency pathway through a semantic-frequency side adapter for multi-scale feature interaction, and employ a prototype-guided, frequency-gated mask decoder to integrate semantic consistency with boundary-aware localization for tampered region prediction. Extensive experiments on OpenSDI and multiple traditional manipulation benchmarks demonstrate state-of-the-art localization performance, strong cross-generator and cross-dataset generalization, and robust performance under common image degradations.

Problem

• image manipulation localization
• micro-macro gap
• diffusion-generated edits
• forensic artifacts
• semantic alignment

Innovation

• frequency-aware
• semantic alignment
• image manipulation localization
• dual-band DCT
• CLIP contrastive learning