A Classifier-Agnostic Zero-Shot Adversarial Attack Detection via CLIP

2026-06-29

AI Summary

Existing adversarial attack detection methods typically rely on attack priors, access to adversarial examples, or internal model information, limiting their applicability in black-box and zero-shot settings. This work proposes A⁴D, a novel framework that introduces CLIP into adversarial detection for the first time. Leveraging CLIP’s sensitivity to non-semantic perturbations and the characteristic shifts in prompt similarity within its multimodal embedding space, A⁴D enables universal detection of arbitrary attacks against any classifier without requiring training or access to the target model. Extensive experiments demonstrate that A⁴D achieves state-of-the-art performance across diverse attacks, datasets, and classifiers, establishing new benchmarks for attack-agnostic and classifier-agnostic detection.

Abstract

Adversarial attacks pose a challenge to the reliability of deep learning models, motivating effective detection methods. Existing techniques often rely on attack-specific assumptions, access to adversarial samples, or knowledge of the underlying classifier (white-box). We propose \textit{$A^4D$ (\textbf{A}ttack- and \textbf{A}rchitecture-\textbf{A}gnostic \textbf{A}dversarial \textbf{D}etector)}, a completely black-box, zero-shot adversarial attack detection framework that utilizes prompt-based similarity scores derived from CLIP. To the best of our knowledge this is the first attempt to utilize CLIP for such a task. The method is based on two key observations: (i) CLIP is sensitive even to small imperceptible non-semantic perturbations; (ii) The shift in CLIP embedding space is not arbitrary and can be used as a robust attack indicator. Experiments across multiple attacks, datasets and classifiers validate that $A^4D$ achieves SOTA detection results in the attack-agnostic and classifier-agnostic setting.

Problem

• adversarial attack detection
• classifier-agnostic
• zero-shot
• black-box
• CLIP

Innovation

• zero-shot
• adversarial attack detection
• CLIP
• black-box
• classifier-agnostic