Universal Adversarial Purification with DDIM Metric Loss for Stable Diffusion

2026-01-12

AI Summary

This work addresses the vulnerability of Stable Diffusion to adversarial attacks targeting its VAE or UNet components, which significantly degrade generation quality and resist existing denoising defenses. To this end, the authors propose the Universal Diffusion Adversarial Purification (UDAP) framework—the first general-purpose adversarial purification method tailored specifically for Stable Diffusion. UDAP leverages DDIM inversion to reveal distinct reconstruction behaviors between clean and adversarial samples, introduces a DDIM-based metric loss to guide the purification process, and incorporates a dynamic iteration adjustment strategy to enhance efficiency. Extensive experiments demonstrate that UDAP achieves strong robustness against diverse attacks—including PID, Anti-DreamBooth, MIST, Anti-DF, and MetaCloak—while maintaining excellent generalization across different model versions and text prompts.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Ad search and search for Web retail
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Stable Diffusion (SD) often produces degraded outputs when the training dataset contains adversarial noise. Adversarial purification offers a promising solution by removing adversarial noise from contaminated data. However, existing purification methods are primarily designed for classification tasks and fail to address SD-specific adversarial strategies, such as attacks targeting the VAE encoder, UNet denoiser, or both. To address the gap in SD security, we propose Universal Diffusion Adversarial Purification (UDAP), a novel framework tailored for defending adversarial attacks targeting SD models. UDAP leverages the distinct reconstruction behaviors of clean and adversarial images during Denoising Diffusion Implicit Models (DDIM) inversion to optimize the purification process. By minimizing the DDIM metric loss, UDAP can effectively remove adversarial noise. Additionally, we introduce a dynamic epoch adjustment strategy that adapts optimization iterations based on reconstruction errors, significantly improving efficiency without sacrificing purification quality. Experiments demonstrate UDAP's robustness against diverse adversarial methods, including PID (VAE-targeted), Anti-DreamBooth (UNet-targeted), MIST (hybrid), and robustness-enhanced variants like Anti-Diffusion (Anti-DF) and MetaCloak. UDAP also generalizes well across SD versions and text prompts, showcasing its practical applicability in real-world scenarios.

Problem

• Stable Diffusion
• adversarial purification
• adversarial attacks
• DDIM
• diffusion models

Innovation

• Universal Adversarial Purification
• DDIM Metric Loss
• Stable Diffusion Security
• Dynamic Epoch Adjustment
• Adversarial Defense