Visual-Friendly Concept Protection via Selective Adversarial Perturbations

2024-08-16

arXiv.org

AI Summary

To address the risk of unauthorized extraction of personalized concepts from diffusion models, this paper proposes a visually friendly concept protection framework. It employs selective adversarial perturbations—injecting low-perceptibility perturbations exclusively into semantically critical regions—to suppress unintended concept leakage. Methodologically, it introduces a novel perturbation generation paradigm that jointly leverages a relaxed optimization objective and Lagrange multiplier-based constrained optimization, achieving an optimal trade-off between protection efficacy and visual fidelity while enabling user-specified target concepts. Extensive experiments across multiple benchmarks demonstrate that the method reduces target concept leakage to below 8%, achieves a protection success rate of 94.7%, and preserves image quality effectively—evidenced by only a 32% decrease in LPIPS (indicating improved perceptual similarity) and a marginal 1.2 increase in FID.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Large-scale security measurements
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Personalized concept generation by tuning diffusion models with a few images raises potential legal and ethical concerns regarding privacy and intellectual property rights. Researchers attempt to prevent malicious personalization using adversarial perturbations. However, previous efforts have mainly focused on the effectiveness of protection while neglecting the visibility of perturbations. They utilize global adversarial perturbations, which introduce noticeable alterations to original images and significantly degrade visual quality. In this work, we propose the Visual-Friendly Concept Protection (VCPro) framework, which prioritizes the protection of key concepts chosen by the image owner through adversarial perturbations with lower perceptibility. To ensure these perturbations are as inconspicuous as possible, we introduce a relaxed optimization objective to identify the least perceptible yet effective adversarial perturbations, solved using the Lagrangian multiplier method. Qualitative and quantitative experiments validate that VCPro achieves a better trade-off between the visibility of perturbations and protection effectiveness, effectively prioritizing the protection of target concepts in images with less perceptible perturbations.

Problem

• Protect privacy and intellectual property in personalized concept generation
• Reduce visible adversarial perturbations while maintaining protection effectiveness
• Balance perturbation visibility and concept protection in diffusion models

Innovation

• Selective adversarial perturbations for concept protection
• Relaxed optimization for minimal perceptibility
• Lagrangian multiplier method for effective perturbation