Dense Feature Interaction Network for Image Inpainting Localization

2024-08-05

IEEE Transactions on Information Forensics and Security

AI Summary

Existing image inpainting detection methods suffer from high false-positive rates and severe false negatives in multi-semantic/multi-scale object localization, and struggle to model boundary artifacts, leading to inaccurate edge localization. To address these issues, we propose the Dense Feature Interaction Network (DeFI-Net), which constructs a multi-stage dense feature pyramid. It enables cross-scale collaborative fusion of low-level edge structures and high-level semantic information via dense skip connections and an adaptive feature re-weighting module. Additionally, we introduce a guidance-aware multi-scale supervision loss to explicitly enhance boundary artifact modeling. Extensive experiments demonstrate that DeFI-Net achieves state-of-the-art performance on seven mainstream inpainting detection benchmarks, significantly reducing both false positives and false negatives while notably improving localization accuracy—especially along inpainted region boundaries.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Image inpainting, the process of filling in missing areas in an image, is a common image editing technique. Inpainting can be used to conceal or alter image contents in malicious manipulation of images, driving the need for research in image inpainting detection. Most existing methods use a basic encoder-decoder structure, which often results in a high number of false positives or misses the inpainted regions, especially when dealing with targets of varying semantics and scales. Additionally, the lack of an effective approach to capture boundary artifacts leads to less accurate edge localization. In this paper, we describe a new method for inpainting detection based on a Dense Feature Interaction Network (DeFI-Net). DeFI-Net uses a novel feature pyramid architecture to capture and amplify multi-scale representations across various stages, thereby improving the detection of image inpainting by better strengthening feature-level interactions. Additionally, the network can adaptively direct the lower-level features, which carry edge and shape information, to refine the localization of manipulated regions while integrating the higher-level semantic features. Using DeFI-Net, we develop a method combining complementary representations to accurately identify inpainted areas. Evaluation on seven image inpainting datasets demonstrates the effectiveness of our approach, which achieves state-of-the-art performance in detecting inpainting across diverse models. Code and models are available at https://github.com/Boombb/DeFI-Net_Inpainting.

Problem

• Detect manipulated inpainted regions in images accurately
• Reduce false positives and misses in inpainting detection
• Improve edge localization by capturing boundary artifacts

Innovation

• Dense Feature Interaction Network for detection
• Novel feature pyramid captures multi-scale representations
• Adaptively integrates edge and semantic features