Vanish into Thin Air: Cross-prompt Universal Adversarial Attacks for SAM2

2025-10-28

AI Summary

This work presents the first systematic robustness evaluation of SAM2 under adversarial attacks, uncovering novel vulnerabilities arising from its prompt-directed guidance and cross-frame semantic entanglement. Method: We propose UAP-SAM2—the first universal adversarial perturbation (UAP) method tailored for video segmentation—featuring a dual-semantic-deviation optimization framework that jointly degrades intra-frame prompt-response semantic consistency and inter-frame semantic coherence. To enhance transferability, we introduce a target-scanning strategy and a randomized region-based prompt allocation mechanism, reducing reliance on specific prompts. Contribution/Results: Evaluated across six benchmark datasets, UAP-SAM2 significantly outperforms existing state-of-the-art adversarial attacks. It provides the first empirical evidence of severe security risks in SAM2 for video understanding tasks, establishing a critical benchmark and offering theoretical insights for future robustness research and defense design.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Recent studies reveal the vulnerability of the image segmentation foundation model SAM to adversarial examples. Its successor, SAM2, has attracted significant attention due to its strong generalization capability in video segmentation. However, its robustness remains unexplored, and it is unclear whether existing attacks on SAM can be directly transferred to SAM2. In this paper, we first analyze the performance gap of existing attacks between SAM and SAM2 and highlight two key challenges arising from their architectural differences: directional guidance from the prompt and semantic entanglement across consecutive frames. To address these issues, we propose UAP-SAM2, the first cross-prompt universal adversarial attack against SAM2 driven by dual semantic deviation. For cross-prompt transferability, we begin by designing a target-scanning strategy that divides each frame into k regions, each randomly assigned a prompt, to reduce prompt dependency during optimization. For effectiveness, we design a dual semantic deviation framework that optimizes a UAP by distorting the semantics within the current frame and disrupting the semantic consistency across consecutive frames. Extensive experiments on six datasets across two segmentation tasks demonstrate the effectiveness of the proposed method for SAM2. The comparative results show that UAP-SAM2 significantly outperforms state-of-the-art (SOTA) attacks by a large margin.

Problem

• Analyzes SAM2's vulnerability to cross-prompt universal adversarial attacks
• Addresses challenges from prompt guidance and frame semantic entanglement
• Proposes dual semantic deviation framework for effective SAM2 attacks

Innovation

• Cross-prompt universal adversarial attack for SAM2
• Target-scanning strategy reduces prompt dependency
• Dual semantic deviation framework distorts semantics