FractalForensics: Proactive Deepfake Detection and Localization via Fractal Watermarks

2025-04-13

AI Summary

To address the challenges of detecting, localizing, and interpreting high-fidelity Deepfakes, this paper proposes a fractal-structure-based active semi-fragile watermarking method. We introduce a parameter-driven fractal watermark generation scheme coupled with a one-way encryption mechanism; design a robustness–sensitivity co-optimized semi-fragile watermarking framework—achieving >98% detection accuracy under common distortions (e.g., JPEG compression, filtering) while maintaining high sensitivity to Deepfake manipulations; and develop an end-to-patch embedding strategy enabling pixel-level tampering localization (average IoU = 92.7%) and visual interpretability. Evaluated on multiple benchmarks, our method significantly outperforms state-of-the-art passive detectors and existing semi-fragile watermarking approaches. To the best of our knowledge, it is the first to unify detection, precise localization, and explainability in a single framework.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Active Learning
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Proactive Deepfake detection via robust watermarks has been raised ever since passive Deepfake detectors encountered challenges in identifying high-quality synthetic images. However, while demonstrating reasonable detection performance, they lack localization functionality and explainability in detection results. Additionally, the unstable robustness of watermarks can significantly affect the detection performance accordingly. In this study, we propose novel fractal watermarks for proactive Deepfake detection and localization, namely FractalForensics. Benefiting from the characteristics of fractals, we devise a parameter-driven watermark generation pipeline that derives fractal-based watermarks and conducts one-way encryption regarding the parameters selected. Subsequently, we propose a semi-fragile watermarking framework for watermark embedding and recovery, trained to be robust against benign image processing operations and fragile when facing Deepfake manipulations in a black-box setting. Meanwhile, we introduce an entry-to-patch strategy that implicitly embeds the watermark matrix entries into image patches at corresponding positions, achieving localization of Deepfake manipulations. Extensive experiments demonstrate satisfactory robustness and fragility of our approach against common image processing operations and Deepfake manipulations, outperforming state-of-the-art semi-fragile watermarking algorithms and passive detectors for Deepfake detection. Furthermore, by highlighting the areas manipulated, our method provides explainability for the proactive Deepfake detection results.

Problem

• Proactive Deepfake detection lacks localization and explainability
• Watermark robustness instability affects detection performance
• Need for robust yet fragile watermarking against manipulations

Innovation

• Fractal watermarks for Deepfake detection
• Semi-fragile watermarking framework
• Entry-to-patch strategy for localization