DisPatch: Disarming Adversarial Patches in Object Detection with Diffusion Models

2025-09-04

AI Summary

Adversarial patch attacks can mislead object detectors, causing missed or false detections, and exhibit diverse, unpredictable attack patterns. This paper proposes the first diffusion-model-based universal defense framework, achieving adaptive robust protection without prior knowledge of the attack. Our method departs from conventional post-processing paradigms by leveraging diffusion models for semantically consistent whole-image reconstruction, coupled with an attention-guided regional comparison mechanism to precisely identify and rectify perturbed regions. Evaluated across multiple detectors (YOLOv5/v8, Faster R-CNN) and mainstream patch attacks (Hide/Create), our approach significantly improves robustness: under Hide attacks, mAP₅₀ rises to 89.3%; under Create attacks, success rate drops to 24.8%; and strong resilience is maintained against adaptive attacks. The framework thus offers a principled, generalizable solution to adversarial patch vulnerabilities in modern object detection systems.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Object detection is fundamental to various real-world applications, such as security monitoring and surveillance video analysis. Despite their advancements, state-of-theart object detectors are still vulnerable to adversarial patch attacks, which can be easily applied to real-world objects to either conceal actual items or create non-existent ones, leading to severe consequences. Given the current diversity of adversarial patch attacks and potential unknown threats, an ideal defense method should be effective, generalizable, and robust against adaptive attacks. In this work, we introduce DISPATCH, the first diffusion-based defense framework for object detection. Unlike previous works that aim to "detect and remove" adversarial patches, DISPATCH adopts a "regenerate and rectify" strategy, leveraging generative models to disarm attack effects while preserving the integrity of the input image. Specifically, we utilize the in-distribution generative power of diffusion models to regenerate the entire image, aligning it with benign data. A rectification process is then employed to identify and replace adversarial regions with their regenerated benign counterparts. DISPATCH is attack-agnostic and requires no prior knowledge of the existing patches. Extensive experiments across multiple detectors and attacks demonstrate that DISPATCH consistently outperforms state-of-the-art defenses on both hiding attacks and creating attacks, achieving the best overall mAP.5 score of 89.3% on hiding attacks, and lowering the attack success rate to 24.8% on untargeted creating attacks. Moreover, it maintains strong robustness against adaptive attacks, making it a practical and reliable defense for object detection systems.

Problem

• Defending object detectors against adversarial patch attacks
• Disarming attack effects without prior knowledge of patches
• Ensuring robustness against diverse and adaptive threats

Innovation

• Diffusion-based defense framework for object detection
• Regenerate and rectify strategy with generative models
• Attack-agnostic approach requiring no prior knowledge