LoRA-Key: User-Centric LoRA Watermarking for Text-to-Image Diffusion Models

2026-05-28

AI Summary

This work addresses the vulnerability of LoRA modules in text-to-image diffusion models to unauthorized replication and the limitations of existing watermarking approaches, which struggle to provide scalable copyright protection without retraining. The authors propose LoRA-Key, a novel framework that introduces the first reusable, plug-and-play watermarking mechanism: a separately trained watermark LoRA module is linearly combined with any target LoRA to embed ownership information, enabling verification without fine-tuning. By integrating VAE latent-space watermark priors, message-conditioned supervision, semantic consistency constraints, and gradient orthogonal projection (GOP) optimization, the method robustly recovers copyright signals under diverse perturbations, downstream fine-tuning, and multi-LoRA compositions, while preserving high-fidelity image generation and stylistic integrity.

Abstract

Low-Rank Adaptation (LoRA) has become a widely used mechanism for customizing text-to-image diffusion models, enabling lightweight modules that are shared, reused, and commercialized as independent assets. This LoRA-centric ecosystem shifts copyright protection from foundation models to distributed LoRA modules, which are easy to copy, redistribute, or reuse without authorization. Existing watermarking methods either protect the base diffusion model or require watermark-aware retraining for each target LoRA, limiting their practicality in open community settings. To address this limitation, we propose LoRA-Key, a user-centric LoRA watermarking framework that treats copyright protection as a reusable ownership key. LoRA-Key encapsulates a recoverable secret message into a standalone user-specific Watermark LoRA, which can be attached to different target LoRAs through training-free linear superposition without per-LoRA retraining or structural modification. To train such a reusable key, we first establish a latent watermark prior in the frozen VAE latent space for robust message embedding and recovery, and then optimize the Watermark LoRA with message-conditioned watermark supervision and semantic consistency constraints. We further introduce Gradient Orthogonal Projection (GOP) to suppress watermark updates that conflict with semantic-preserving directions, reducing interference with generation fidelity and downstream style adaptation. Extensive experiments show that LoRA-Key provides lightweight plug-and-play copyright protection while preserving generation quality and style fidelity, and maintains robust ownership verification under image-level distortions, downstream fine-tuning, and multi-LoRA composition.

Problem

• LoRA
• watermarking
• copyright protection
• text-to-image diffusion models
• ownership verification

Innovation

• LoRA watermarking
• user-centric copyright
• training-free superposition
• gradient orthogonal projection
• latent watermark prior