🤖 AI Summary
Existing work has not yet investigated the adversarial robustness of SAM3 in open-vocabulary concept segmentation, nor has it developed a universal attack method targeting its joint concept–image system. This work proposes Universal Concept Disruption (UCD), the first approach enabling end-to-end, universal cross-concept adversarial attacks on SAM3. UCD learns a single bounded perturbation that simultaneously disrupts textual conditioning inputs, visual feature consistency, existence gating scores, and mask spatial validity. Notably, this perturbation generalizes across datasets, model variants (e.g., SAM3.1), and video inference without re-optimization. Experiments demonstrate that UCD substantially degrades model performance, reducing average mask AP from 59.43 to 18.73 and cgF1 from 50.32 to 20.49, thereby confirming its strong transferability and effectiveness.
📝 Abstract
SAM3 extends promptable segmentation from geometry-driven mask prediction to open-vocabulary concept segmentation, where a text-conditioned grounding model decides whether a concept is present and segments all matching instances. While this presence-gated design improves concept-level prediction, its adversarial robustness remains unexplored. In this paper, we introduce Universal Concept Disruption (UCD), the first universal cross-concept adversarial attack tailored to SAM3 image segmentation. UCD learns a single bounded image perturbation from (image, noun-phrase) pairs and attacks SAM3 as an integrated concept-grounding system. It jointly disrupts the text-conditioned input path, maximizes divergence in prompt-shared visual features, suppresses the final presence-gated concept scores, and corrupts the spatial validity of retained masks through area collapse and clean-mask Dice disruption. Across SACo-Gold, LVIS, RefCOCO, PhraseCut, and OpenImages datasets, UCD consistently outperforms all baselines under a matched evaluation protocol, reducing average mask AP from 59.43 to 18.73 and average cgF1 from 50.32 to 20.49. The learned perturbation also transfers to SAM3.1 and to SAM3 video inference without re-optimization, while prompt ensembling, lightweight head fine-tuning, and temporal filtering provide limited recovery.