HMARK: Radioactive Multi-Bit Semantic-Latent Watermarking for Diffusion Models

2025-11-26

AI Summary

To address the ambiguous copyright status of training data in generative diffusion models, this paper proposes a radiative multi-bit watermarking scheme operating in the semantic latent space (h-space). Unlike conventional pixel-domain watermarking, our approach is the first to exploit the interpretability of diffusion models’ semantic latent representations for watermark encoding—ensuring perceptual invisibility while enabling reliable watermark propagation (“radiation”) from inputs to generated outputs, strong robustness against diverse perturbations, and high information capacity (≥16 bits). We validate the end-to-end watermark embedding–propagation–recovery pipeline via LoRA fine-tuning and adversarial training. Experiments demonstrate exceptional performance: 98.57% watermark detection accuracy, 95.07% bit-level recovery rate, 100% recall, and an AUC of 1.0—substantially outperforming state-of-the-art methods.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Modern generative diffusion models rely on vast training datasets, often including images with uncertain ownership or usage rights. Radioactive watermarks -- marks that transfer to a model's outputs -- can help detect when such unauthorized data has been used for training. Moreover, aside from being radioactive, an effective watermark for protecting images from unauthorized training also needs to meet other existing requirements, such as imperceptibility, robustness, and multi-bit capacity. To overcome these challenges, we propose HMARK, a novel multi-bit watermarking scheme, which encodes ownership information as secret bits in the semantic-latent space (h-space) for image diffusion models. By leveraging the interpretability and semantic significance of h-space, ensuring that watermark signals correspond to meaningful semantic attributes, the watermarks embedded by HMARK exhibit radioactivity, robustness to distortions, and minimal impact on perceptual quality. Experimental results demonstrate that HMARK achieves 98.57% watermark detection accuracy, 95.07% bit-level recovery accuracy, 100% recall rate, and 1.0 AUC on images produced by the downstream adversarial model finetuned with LoRA on watermarked data across various types of distortions.

Problem

• Detects unauthorized training data in diffusion models
• Encodes ownership bits in semantic-latent space for protection
• Ensures watermarks are radioactive, robust, and imperceptible

Innovation

• Watermarking in semantic-latent space for diffusion models
• Encoding multi-bit ownership information as secret bits
• Ensuring radioactivity, robustness, and minimal perceptual impact