ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization

2026-06-23

AI Summary

This work addresses the limitations of existing multimodal large language model (MLLM)-based image tampering localization methods, which rely on external segmentation decoders and suffer from spatial signal dilution and insufficient semantic priors, leading to suboptimal localization accuracy. The authors reformulate tampering localization as an autoregressive sequence generation task, enabling end-to-end mask prediction through direct output of spatially aligned token sequences without intermediate supervision. To mitigate gradient discontinuities in deterministic decoding, they introduce a Token Splatting Decoder and further propose a Hierarchical Expert Fusion module to integrate multi-scale forensic features, compensating for the MLLM’s lack of built-in forensic priors. The method significantly outperforms current MLLM-based approaches across six benchmarks, slightly surpasses specialized forensic models, and demonstrates enhanced robustness to perturbations.

Abstract

Multi-modal Large Language Models (MLLMs) offer powerful reasoning for forensic tasks, yet existing approaches utilizing exogenous segmentation decoders often suffer from suboptimal localization. The reliance on stitched pipelines introduces information bottlenecks during backpropagation, which dilutes spatial signals and is limited by semantic priors of the segmentor. To address these limitations, we propose ForensicsTok, which reformulates image manipulation localization as an autoregressive sequence generation task. ForensicsTok directly generates spatially grounded token sequences, enabling precise mask prediction without intermediary supervision. Specifically, we introduce a Token Splatting Decoder (TSD) to map tokens to binary masks via codebook-aware code smoothing, which mitigates sharp gradients from deterministic detokenizers. Furthermore, to capture diverse tampering clues, we propose a Hierarchical Expert Fusion (HEF) module that injects multi-scale features from a forensic expert model. This unified architecture effectively compensates for the lack of forensic priors in standard MLLMs. Extensive experiments on six benchmarks show that ForensicsTok substantially improves over existing MLLM-based baselines and slightly improves over strong forensic expert baselines, while exhibiting stronger robustness to perturbations.

Problem

• image tampering localization
• multi-modal large language models
• segmentation decoder
• spatial signal
• forensic priors

Innovation

• tokenized modeling
• image tampering localization
• autoregressive sequence generation
• Token Splatting Decoder
• Hierarchical Expert Fusion