SARIF: Segment Anything for Robust Image Forensics

2026-06-19

AI Summary

Existing image forgery localization methods exhibit limited generalization when confronted with diverse manipulation types and domain shifts. This work proposes a novel architecture based on the Segment Anything Model (SAM) that leverages dual encoders to extract residual forgery traces and integrates a patch-level prompting mechanism with a feedback-guided mask decoder to enable fully automatic segmentation of forged regions. By effectively harnessing SAM’s strong generalization capabilities—a first in the image forensics domain—the proposed method achieves substantial performance gains across multiple standard benchmarks in cross-dataset evaluation and demonstrates remarkable robustness against common image perturbations.

Abstract

Image forgery localization remains challenging due to diverse manipulation techniques and distribution shifts. Existing forgery localization models achieve high accuracy on benchmarks but often struggle with cross-domain generalization and robustness. In this paper, we propose SARIF (Segment Anything for Robust Image Forensics), a framework that leverages the Segment Anything Model (SAM), which has a promptable architecture and strong generalization ability. SARIF introduces a feedback-guided mask decoder and a dual-encoder design that extracts forgery-specific information to capture forensic traces while exploiting the SAM architecture. To localize manipulated regions, we design a block-wise prompting mechanism that derives forgery-specific cues from residual features between an adapted encoder and its frozen counterpart. These features are fused with the previous mask prompt to drive a feedback-based mask refinement process, enabling automatic forgery segmentation without manual input. Extensive experiments on standard forgery-localization benchmarks show that SARIF achieves strong average cross-dataset performance and robustness to common image corruptions.

Problem

• image forgery localization
• cross-domain generalization
• robustness
• distribution shifts
• manipulation techniques

Innovation

• Segment Anything Model
• image forgery localization
• dual-encoder architecture
• feedback-guided mask refinement
• block-wise prompting