Two Heads Are Better Than One: Averaging along Fine-Tuning to Improve Targeted Transferability

2024-12-30

AI Summary

Weak cross-model transferability in targeted adversarial attacks and the neglect of optimization trajectory information in existing fine-tuning methods motivate this work. We propose Trajectory Averaging (TrajAvg), a novel mechanism that, during feature-space fine-tuning, samples and adaptively weights intermediate adversarial examples along the optimization path, then pulls them toward the center of flat regions on the loss surface—thereby enhancing targeted transfer robustness. TrajAvg is the first method to explicitly model the fine-tuning trajectory as a critical signal for improving transferability and is compatible with mainstream targeted attacks (e.g., Targ-IG, Targ-CW). Extensive experiments on ImageNet against diverse target models demonstrate that TrajAvg achieves an average 12.7% improvement in targeted transfer success rate, significantly outperforming state-of-the-art methods. The code is publicly available.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

With much longer optimization time than that of untargeted attacks notwithstanding, the transferability of targeted attacks is still far from satisfactory. Recent studies reveal that fine-tuning an existing adversarial example (AE) in feature space can efficiently boost its targeted transferability. However, existing fine-tuning schemes only utilize the endpoint and ignore the valuable information in the fine-tuning trajectory. Noting that the vanilla fine-tuning trajectory tends to oscillate around the periphery of a flat region of the loss surface, we propose averaging over the fine-tuning trajectory to pull the crafted AE towards a more centered region. We compare the proposed method with existing fine-tuning schemes by integrating them with state-of-the-art targeted attacks in various attacking scenarios. Experimental results uphold the superiority of the proposed method in boosting targeted transferability. The code is available at github.com/zengh5/Avg_FT.

Problem

• Adversarial Samples
• Targeted Attacks
• Effectiveness Improvement

Innovation

• Adversarial Samples
• Flattened Loss Landscape
• Targeted Attack Generalization