Reinforced Multi-teacher Knowledge Distillation for Efficient General Image Forgery Detection and Localization

2025-04-07

AI Summary

To address poor generalization across diverse image forgeries—such as copy-move, splicing, and inpainting—in real-world scenarios, this paper proposes a通用 Image Forgery Detection and Localization (IFDL) framework. Methodologically, it introduces two key innovations: (1) Reinforcement Learning-driven Dynamic Teacher Selection (Re-DTS), a novel mechanism enabling adaptive knowledge distillation from multiple teacher models to a student model; and (2) Cue-Net, an encoder-decoder architecture integrating ConvNeXt-UperNet with an edge-aware module to enhance modeling of forgery boundaries. Evaluated on multiple emerging forgery benchmarks, the proposed method achieves state-of-the-art performance in both detection accuracy and fine-grained localization, while demonstrating strong generalization to unseen forgery types.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Measurement, analysis, and circumvention of Web censorship

Abstract

Image forgery detection and localization (IFDL) is of vital importance as forged images can spread misinformation that poses potential threats to our daily lives. However, previous methods still struggled to effectively handle forged images processed with diverse forgery operations in real-world scenarios. In this paper, we propose a novel Reinforced Multi-teacher Knowledge Distillation (Re-MTKD) framework for the IFDL task, structured around an encoder-decoder extbf{C}onvNeXt- extbf{U}perNet along with extbf{E}dge-Aware Module, named Cue-Net. First, three Cue-Net models are separately trained for the three main types of image forgeries, i.e., copy-move, splicing, and inpainting, which then serve as the multi-teacher models to train the target student model with Cue-Net through self-knowledge distillation. A Reinforced Dynamic Teacher Selection (Re-DTS) strategy is developed to dynamically assign weights to the involved teacher models, which facilitates specific knowledge transfer and enables the student model to effectively learn both the common and specific natures of diverse tampering traces. Extensive experiments demonstrate that, compared with other state-of-the-art methods, the proposed method achieves superior performance on several recently emerged datasets comprised of various kinds of image forgeries.

Problem

• Detect and localize diverse image forgeries efficiently
• Handle real-world forged images with multiple operations
• Transfer knowledge from multi-teacher models effectively

Innovation

• Reinforced Multi-teacher Knowledge Distillation framework
• ConvNeXt-UperNet with Edge-Aware Module
• Reinforced Dynamic Teacher Selection strategy