Boosting the Local Invariance for Better Adversarial Transferability

2025-03-08

AI Summary

This work addresses the weak cross-model transferability of adversarial examples in transfer-based adversarial attacks. We first reveal that the local translation invariance of adversarial perturbations—i.e., their consistency under small spatial shifts—is positively correlated with transferability, identifying it as a key intrinsic mechanism governing transferability. To exploit this insight, we propose LI-Boost, a general-purpose enhancement framework that jointly optimizes perturbations via gradient descent while enforcing local translation invariance through a consistency constraint. LI-Boost is model-agnostic and attack-agnostic, requiring no knowledge of target model architecture or attack paradigm. Evaluated on ImageNet, it significantly improves transferability across diverse architectures (CNNs and Vision Transformers) and robust defenses (e.g., feature squeezing, input randomization). Compatible with various white-box attacks—including gradient-based, transformation-based, and ensemble methods—LI-Boost substantially boosts black-box attack success rates, even against strongly defended models. This work provides both a novel theoretical interpretation of transferability and a practical, broadly applicable tool for transfer attacks.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Transfer-based attacks pose a significant threat to real-world applications by directly targeting victim models with adversarial examples generated on surrogate models. While numerous approaches have been proposed to enhance adversarial transferability, existing works often overlook the intrinsic relationship between adversarial perturbations and input images. In this work, we find that adversarial perturbation often exhibits poor translation invariance for a given clean image and model, which is attributed to local invariance. Through empirical analysis, we demonstrate that there is a positive correlation between the local invariance of adversarial perturbations w.r.t. the input image and their transferability across different models. Based on this finding, we propose a general adversarial transferability boosting technique called Local Invariance Boosting approach (LI-Boost). Extensive experiments on the standard ImageNet dataset demonstrate that LI-Boost could significantly boost various types of transfer-based attacks (e.g., gradient-based, input transformation-based, model-related, advanced objective function, ensemble, etc.) on CNNs, ViTs, and defense mechanisms. Our approach presents a promising direction for future research in improving adversarial transferability across different models.

Problem

• Enhance adversarial transferability across different models
• Address poor translation invariance in adversarial perturbations
• Propose Local Invariance Boosting (LI-Boost) technique

Innovation

• Enhances adversarial transferability via local invariance
• Proposes LI-Boost technique for boosting transfer-based attacks
• Improves attack effectiveness across CNNs, ViTs, defenses