Secure and Efficient Watermarking for Latent Diffusion Models in Model Distribution Scenarios

2025-02-18

AI Summary

To address critical limitations of latent diffusion models (LDMs) in copyright protection—namely, watermark leakage, vulnerability to evasion attacks, poor verification robustness, and low embedding efficiency—this paper proposes a novel watermarking framework tailored for LDM-based distribution. Methodologically, it introduces the first dual-security constraint: “watermark randomness” and “strong watermark–model coupling,” decoupling watermark injection from security module training and enabling lightweight verification via watermark statistical distribution modeling. Technically, the framework integrates VAE fine-tuning, constrained optimization, and robust detection modeling. Experiments across ten common image processing operations and adversarial attacks demonstrate that our method achieves an average 12.7% higher watermark detection accuracy than six state-of-the-art baselines, reduces training overhead by 38%, and completely resists known evasion attacks—thereby significantly enhancing the security, efficiency, and robustness of copyright protection in LDMs.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Latent diffusion models have exhibited considerable potential in generative tasks. Watermarking is considered to be an alternative to safeguard the copyright of generative models and prevent their misuse. However, in the context of model distribution scenarios, the accessibility of models to large scale of model users brings new challenges to the security, efficiency and robustness of existing watermark solutions. To address these issues, we propose a secure and efficient watermarking solution. A new security mechanism is designed to prevent watermark leakage and watermark escape, which considers watermark randomness and watermark-model association as two constraints for mandatory watermark injection. To reduce the time cost of training the security module, watermark injection and the security mechanism are decoupled, ensuring that fine-tuning VAE only accomplishes the security mechanism without the burden of learning watermark patterns. A watermark distribution-based verification strategy is proposed to enhance the robustness against diverse attacks in the model distribution scenarios. Experimental results prove that our watermarking consistently outperforms existing six baselines on effectiveness and robustness against ten image processing attacks and adversarial attacks, while enhancing security in the distribution scenarios.

Problem

• Secure watermarking for latent diffusion models
• Prevent watermark leakage and escape
• Enhance robustness against diverse attacks

Innovation

• Prevents watermark leakage and escape
• Decouples watermark injection and security
• Enhances robustness against diverse attacks