DiffMark: Diffusion-based Robust Watermark Against Deepfakes

2025-07-02

AI Summary

To address the insufficient robustness of existing watermarking methods against Deepfake attacks, this paper proposes DiffMark, a diffusion-based robust watermarking framework. Methodologically: (1) a timestep-dependent weighting mechanism dynamically modulates facial-conditioning guidance strength; (2) a cross-information fusion module adaptively integrates watermark and image features; (3) a frozen autoencoder simulates Deepfake attacks, while an adversarial deepfake-guided sampling strategy enhances watermark resilience to tampering. Technically, DiffMark unifies diffusion modeling, cross-attention, learnable embedding tables, and attack-aware training. Experiments demonstrate that DiffMark achieves significantly higher watermark retention rates and identification accuracy than state-of-the-art methods under diverse Deepfake generation attacks—including face-swapping, reenactment, and morphing—thereby enabling reliable authenticity verification and provenance tracing.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance

Abstract

Deepfakes pose significant security and privacy threats through malicious facial manipulations. While robust watermarking can aid in authenticity verification and source tracking, existing methods often lack the sufficient robustness against Deepfake manipulations. Diffusion models have demonstrated remarkable performance in image generation, enabling the seamless fusion of watermark with image during generation. In this study, we propose a novel robust watermarking framework based on diffusion model, called DiffMark. By modifying the training and sampling scheme, we take the facial image and watermark as conditions to guide the diffusion model to progressively denoise and generate corresponding watermarked image. In the construction of facial condition, we weight the facial image by a timestep-dependent factor that gradually reduces the guidance intensity with the decrease of noise, thus better adapting to the sampling process of diffusion model. To achieve the fusion of watermark condition, we introduce a cross information fusion (CIF) module that leverages a learnable embedding table to adaptively extract watermark features and integrates them with image features via cross-attention. To enhance the robustness of the watermark against Deepfake manipulations, we integrate a frozen autoencoder during training phase to simulate Deepfake manipulations. Additionally, we introduce Deepfake-resistant guidance that employs specific Deepfake model to adversarially guide the diffusion sampling process to generate more robust watermarked images. Experimental results demonstrate the effectiveness of the proposed DiffMark on typical Deepfakes. Our code will be available at https://github.com/vpsg-research/DiffMark.

Problem

• Develop robust watermarking against Deepfake facial manipulations
• Enhance watermark fusion using diffusion models and adaptive features
• Improve watermark robustness via adversarial Deepfake-resistant guidance

Innovation

• Diffusion model for watermark fusion
• Cross-attention based watermark feature integration
• Deepfake-resistant guidance for robustness