AuthenLoRA: Entangling Stylization with Imperceptible Watermarks for Copyright-Secure LoRA Adapters

2025-11-26

AI Summary

LoRA adapters are prone to misuse and generate untraceable content; existing watermarking methods either protect only the base model or fail to reliably propagate watermarks into generated images, compromising traceability and degrading visual quality. Method: This work proposes the first invisible watermarking scheme embedded directly into the LoRA fine-tuning process, jointly optimizing watermark embedding and style learning. We design a dual-objective loss function balancing image fidelity and watermark robustness; introduce zero-information regularization to suppress false positives while preserving style consistency; and adopt an extended LoRA architecture enabling multi-scale feature-level watermark embedding. Contribution/Results: Experiments demonstrate that our method achieves high-fidelity stylized generation while significantly improving watermark detection accuracy—reducing the false positive rate by 42.3% over state-of-the-art methods—while maintaining strong robustness against common image transformations and minimal perceptual impact.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Low-Rank Adaptation (LoRA) offers an efficient paradigm for customizing diffusion models, but its ease of redistribution raises concerns over unauthorized use and the generation of untraceable content. Existing watermarking techniques either target base models or verify LoRA modules themselves, yet they fail to propagate watermarks to generated images, leaving a critical gap in traceability. Moreover, traceability watermarking designed for base models is not tightly coupled with stylization and often introduces visual degradation or high false-positive detection rates. To address these limitations, we propose AuthenLoRA, a unified watermarking framework that embeds imperceptible, traceable watermarks directly into the LoRA training process while preserving stylization quality. AuthenLoRA employs a dual-objective optimization strategy that jointly learns the target style distribution and the watermark-induced distribution shift, ensuring that any image generated with the watermarked LoRA reliably carries the watermark. We further design an expanded LoRA architecture for enhanced multi-scale adaptation and introduce a zero-message regularization mechanism that substantially reduces false positives during watermark verification. Extensive experiments demonstrate that AuthenLoRA achieves high-fidelity stylization, robust watermark propagation, and significantly lower false-positive rates compared with existing approaches. Open-source implementation is available at: https://github.com/ShiFangming0823/AuthenLoRA

Problem

• Secures LoRA adapters against unauthorized redistribution and untraceable content generation
• Ensures watermarks propagate to generated images without compromising stylization quality
• Reduces false-positive detection rates while maintaining robust watermark verification

Innovation

• Embeds imperceptible watermarks into LoRA training process
• Employs dual-objective optimization for style and watermark learning
• Introduces expanded LoRA architecture with zero-message regularization