SCA: Highly Efficient Semantic-Consistent Unrestricted Adversarial Attack

2024-10-03

arXiv.org

AI Summary

Deep neural networks are vulnerable to adversarial attacks in safety-critical applications. While unrestricted adversarial attack methods improve sample realism, they often introduce severe semantic distortions and suffer from low sampling efficiency. To address this, we propose Semantic-Consistent Adversarial Editing (SCAE), the first framework that explicitly defines and generates semantic-consistent adversarial examples. SCAE leverages multimodal large language models (MLLMs) to provide fine-grained semantic priors that guide the reverse diffusion process; it further introduces an edit-friendly noise map to inject semantic constraints into the accelerated DPM-Solver++ sampling pipeline. Experiments demonstrate that SCAE achieves 12× faster generation than state-of-the-art methods, significantly reduces semantic distortion, and simultaneously enhances visual fidelity and attack success rate. Our approach establishes a new paradigm for robustness evaluation in high-stakes scenarios.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Security and Privacy: Large-scale security measurements

Abstract

Deep neural network based systems deployed in sensitive environments are vulnerable to adversarial attacks. Unrestricted adversarial attacks typically manipulate the semantic content of an image (e.g., color or texture) to create adversarial examples that are both effective and photorealistic. Recent works have utilized the diffusion inversion process to map images into a latent space, where high-level semantics are manipulated by introducing perturbations. However, they often results in substantial semantic distortions in the denoised output and suffers from low efficiency. In this study, we propose a novel framework called Semantic-Consistent Unrestricted Adversarial Attacks (SCA), which employs an inversion method to extract edit-friendly noise maps and utilizes Multimodal Large Language Model (MLLM) to provide semantic guidance throughout the process. Under the condition of rich semantic information provided by MLLM, we perform the DDPM denoising process of each step using a series of edit-friendly noise maps, and leverage DPM Solver++ to accelerate this process, enabling efficient sampling with semantic consistency. Compared to existing methods, our framework enables the efficient generation of adversarial examples that exhibit minimal discernible semantic changes. Consequently, we for the first time introduce Semantic-Consistent Adversarial Examples (SCAE). Extensive experiments and visualizations have demonstrated the high efficiency of SCA, particularly in being on average 12 times faster than the state-of-the-art attacks. Our research can further draw attention to the security of multimedia information.

Problem

• Addresses semantic distortions in unrestricted adversarial attacks
• Improves efficiency of generating photorealistic adversarial examples
• Ensures semantic consistency using MLLM guidance and DPM Solver++

Innovation

• Uses inversion method for edit-friendly noise maps
• Employs MLLM for semantic guidance
• Accelerates process with DPM Solver++