SEAL: Entangled White-box Watermarks on Low-Rank Adaptation

2025-01-16

AI Summary

To address the lack of copyright protection for LoRA weights and the insufficient robustness of existing steganographic techniques, this paper proposes SEAL—the first general-purpose white-box watermarking framework for LoRA-adapted models. Its core innovation lies in embedding an immutable “passport” matrix into trainable LoRA parameters, enabling end-to-end watermark-model fusion via an implicit parameter entanglement mechanism that incurs no additional loss. SEAL introduces no supervision-based loss, imposes zero performance overhead, and natively supports text, vision, and multimodal tasks. Experiments demonstrate that SEAL preserves original model performance perfectly across diverse downstream tasks—including commonsense reasoning, instruction tuning, and text-to-image generation—with accuracy degradation below 0.1% (ΔAcc < 0.1%). Moreover, it exhibits strong robustness against removal, obfuscation, and ambiguity attacks, achieving watermark extraction accuracy exceeding 99.7%.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Data transparency and provenance

Abstract

Recently, LoRA and its variants have become the de facto strategy for training and sharing task-specific versions of large pretrained models, thanks to their efficiency and simplicity. However, the issue of copyright protection for LoRA weights, especially through watermark-based techniques, remains underexplored. To address this gap, we propose SEAL (SEcure wAtermarking on LoRA weights), the universal whitebox watermarking for LoRA. SEAL embeds a secret, non-trainable matrix between trainable LoRA weights, serving as a passport to claim ownership. SEAL then entangles the passport with the LoRA weights through training, without extra loss for entanglement, and distributes the finetuned weights after hiding the passport. When applying SEAL, we observed no performance degradation across commonsense reasoning, textual/visual instruction tuning, and text-to-image synthesis tasks. We demonstrate that SEAL is robust against a variety of known attacks: removal, obfuscation, and ambiguity attacks.

Problem

• LoRA Weight Protection
• Watermarking Techniques
• Model Ownership Verification

Innovation

• SEAL
• LoRA Weight Tagging
• Secure Model Ownership Verification