Disrupting Semantic and Abstract Features for Better Adversarial Transferability

2025-07-21

AI Summary

Existing feature-level adversarial attacks overly rely on semantic information modeling while neglecting CNNs’ strong preference for high-frequency abstract features (e.g., textures and edges), thereby limiting black-box transferability. To address this, we propose SAFER—the first feature-level attack that jointly models semantic and high-frequency abstract features to enhance transferability. Its core innovation lies in a dual-path mixing strategy: BLOCKMIX operates in the spatial domain, while SELF-MIX operates in the frequency domain; together, they adaptively construct a critical feature weight matrix that fuses semantic and high-frequency responses. Extensive experiments on ImageNet demonstrate that SAFER significantly outperforms state-of-the-art feature-level attacks, achieving an average 8.2% improvement in black-box transfer success rate across diverse target models. This validates the effectiveness of co-optimizing semantic and high-frequency features for improving adversarial transfer robustness.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Large-scale security measurements

Abstract

Adversarial examples pose significant threats to deep neural networks (DNNs), and their property of transferability in the black-box setting has led to the emergence of transfer-based attacks, making it feasible to target real-world applications employing DNNs. Among them, feature-level attacks, where intermediate features are perturbed based on feature importance weight matrix computed from transformed images, have gained popularity. In this work, we find that existing feature-level attacks primarily manipulate the semantic information to derive the weight matrix. Inspired by several works that find CNNs tend to focus more on high-frequency components (a.k.a. abstract features, e.g., texture, edge, etc.), we validate that transforming images in the high-frequency space also improves transferability. Based on this finding, we propose a balanced approach called Semantic and Abstract FEatures disRuption (SAFER). Specifically, SAFER conducts BLOCKMIX on the input image and SELF-MIX on the frequency spectrum when computing the weight matrix to highlight crucial features. By using such a weight matrix, we can direct the attacker to disrupt both semantic and abstract features, leading to improved transferability. Extensive experiments on the ImageNet dataset also demonstrate the effectiveness of our method in boosting adversarial transferability.

Problem

• Improving adversarial transferability in black-box attacks
• Balancing disruption of semantic and abstract features
• Enhancing feature-level attacks with frequency transformations

Innovation

• Disrupts semantic and abstract features simultaneously
• Uses BLOCKMIX and SELF-MIX transformations
• Improves adversarial transferability via frequency space