DiffuseTrace: A Transparent and Flexible Watermarking Scheme for Latent Diffusion Model

2024-05-04

arXiv.org

AI Summary

Copyright protection for content generated by latent diffusion models (LDMs) remains challenging—existing post-hoc watermarking methods are vulnerable to generative attacks and struggle to balance image fidelity with watermark robustness. Method: This paper proposes a plug-and-play, fine-tuning-free semantic watermarking method operating directly in the LDM latent space. It embeds multi-bit watermarks into latent representations via semantic-aware encoder-decoder modules, jointly optimized with adversarial robust training to minimize a differentiable watermark loss. Contribution/Results: Our approach breaks the conventional fidelity–robustness trade-off. Under eight conventional image distortions and three generative watermark-removal attacks, it achieves 99% watermark detection accuracy and >94% attribution accuracy—significantly outperforming state-of-the-art methods while preserving visual quality.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Latent Diffusion Models (LDMs) enable a wide range of applications but raise ethical concerns regarding illegal utilization. Adding watermarks to generative model outputs is a vital technique employed for copyright tracking and mitigating potential risks associated with Artificial Intelligence (AI)-generated contents. However, post-processed watermarking methods are unable to withstand generative watermark attacks and there exists a trade-off between image fidelity and watermark strength. Therefore, we propose a novel technique called DiffuseTrace. DiffuseTrace does not rely on fine-tuning of the diffusion model components. The multi-bit watermark is a embedded into the image space semantically without compromising image quality. The watermark component can be utilized as a plug-in in arbitrary diffusion models. We validate through experiments the effectiveness and flexibility of DiffuseTrace. Under 8 types of image processing watermark attacks and 3 types of generative watermark attacks, DiffuseTrace maintains watermark detection rate of 99% and attribution accuracy of over 94%.

Problem

• Addresses ethical concerns in Latent Diffusion Models usage
• Improves watermark robustness against generative attacks
• Ensures high image fidelity with embedded watermarks

Innovation

• Multi-bit watermark embedded semantically
• Plug-in for arbitrary diffusion models
• High detection rate under attacks