AdvLogo: Adversarial Patch Attack against Object Detectors based on Diffusion Models

2024-09-11

arXiv.org

AI Summary

To address the challenge of balancing attack efficacy and visual quality in adversarial patch attacks against object detectors, this paper proposes the first diffusion model-based, semantics-guided adversarial patch generation framework. Departing from the semantic subspace hypothesis, our method jointly perturbs the latent representation and unconditional embedding at the final denoising step. We further introduce Fourier-domain optimization to mitigate distribution shift and enhance perceptual naturalness. Evaluated on COCO and PASCAL VOC, our approach achieves over 85% black-box and gray-box attack success rates against mainstream detectors—including YOLOv5 and Faster R-CNN—while maintaining high visual fidelity: PSNR > 32 dB and LPIPS < 0.15. These results significantly outperform existing state-of-the-art methods, demonstrating superior trade-offs between attack effectiveness and imperceptibility.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Search and Optimization: Adversarial Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

With the rapid development of deep learning, object detectors have demonstrated impressive performance; however, vulnerabilities still exist in certain scenarios. Current research exploring the vulnerabilities using adversarial patches often struggles to balance the trade-off between attack effectiveness and visual quality. To address this problem, we propose a novel framework of patch attack from semantic perspective, which we refer to as AdvLogo. Based on the hypothesis that every semantic space contains an adversarial subspace where images can cause detectors to fail in recognizing objects, we leverage the semantic understanding of the diffusion denoising process and drive the process to adversarial subareas by perturbing the latent and unconditional embeddings at the last timestep. To mitigate the distribution shift that exposes a negative impact on image quality, we apply perturbation to the latent in frequency domain with the Fourier Transform. Experimental results demonstrate that AdvLogo achieves strong attack performance while maintaining high visual quality.

Problem

• Addresses vulnerabilities in object detectors using adversarial patches.
• Balances attack effectiveness and visual quality in adversarial attacks.
• Proposes AdvLogo framework leveraging semantic understanding for robust attacks.

Innovation

• Semantic-based adversarial patch attack framework
• Diffusion models for adversarial subspace exploration
• Frequency domain perturbation using Fourier Transform