Everywhere Attack: Attacking Locally and Globally to Boost Targeted Transferability

2025-01-01

AI Summary

To address the weak transferability of targeted adversarial examples in black-box settings, this paper proposes a global-local collaborative attack paradigm: input images are partitioned into non-overlapping patches, and diverse local objectives are optimized in parallel over each patch to mitigate attention misalignment between surrogate and target models. We introduce the novel “attack-everywhere” mechanism—replacing the conventional single high-confidence global target with joint optimization of multiple region-specific local targets—rendering the method model-agnostic and plug-and-play. Building upon foundational attacks (e.g., Logit/MI-FGSM), we construct an ensemble-enhanced framework. On ImageNet, our approach comprehensively outperforms state-of-the-art targeted attacks, achieving 28.8%–300% higher Logit-based transfer success rates. Furthermore, strong transfer robustness is empirically validated on the Google Cloud Vision API—a real-world black-box service.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

Adversarial examples' (AE) transferability refers to the phenomenon that AEs crafted with one surrogate model can also fool other models. Notwithstanding remarkable progress in untargeted transferability, its targeted counterpart remains challenging. This paper proposes an everywhere scheme to boost targeted transferability. Our idea is to attack a victim image both globally and locally. We aim to optimize 'an army of targets' in every local image region instead of the previous works that optimize a high-confidence target in the image. Specifically, we split a victim image into non-overlap blocks and jointly mount a targeted attack on each block. Such a strategy mitigates transfer failures caused by attention inconsistency between surrogate and victim models and thus results in stronger transferability. Our approach is method-agnostic, which means it can be easily combined with existing transferable attacks for even higher transferability. Extensive experiments on ImageNet demonstrate that the proposed approach universally improves the state-of-the-art targeted attacks by a clear margin, e.g., the transferability of the widely adopted Logit attack can be improved by 28.8%-300%.We also evaluate the crafted AEs on a real-world platform: Google Cloud Vision. Results further support the superiority of the proposed method.

Problem

• Adversarial Samples
• Transferability
• Targeted Attacks

Innovation

• Comprehensive Attack
• Transferability Enhancement
• Targeted Perturbation