Exploring Semantic-constrained Adversarial Example with Instruction Uncertainty Reduction

2025-10-27

AI Summary

Existing semantic-constrained adversarial example (SemanticAE) generation methods suffer from limited transferability and effectiveness, as they neglect semantic uncertainties inherent in natural language instructions—such as referential diversity, descriptive incompleteness, and boundary ambiguity. To address this, we propose InSUR, a multidimensional instruction uncertainty reduction framework that enables reference-free, semantic-constrained 3D adversarial sample generation for the first time. Our core contributions include: (i) a residual-driven attack direction stabilization mechanism; (ii) context-aware scene constraint encoding; and (iii) a semantic abstraction evaluation enhancement module. InSUR integrates ResAdv-DDIM sampling, guided masking, and multi-step diffusion optimization. Extensive experiments demonstrate that InSUR significantly improves cross-model and cross-domain (2D/3D) transfer attack success rates, while ensuring robust adaptability and generation stability.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recently, semantically constrained adversarial examples (SemanticAE), which are directly generated from natural language instructions, have become a promising avenue for future research due to their flexible attacking forms. To generate SemanticAEs, current methods fall short of satisfactory attacking ability as the key underlying factors of semantic uncertainty in human instructions, such as referring diversity, descriptive incompleteness, and boundary ambiguity, have not been fully investigated. To tackle the issues, this paper develops a multi-dimensional instruction uncertainty reduction (InSUR) framework to generate more satisfactory SemanticAE, i.e., transferable, adaptive, and effective. Specifically, in the dimension of the sampling method, we propose the residual-driven attacking direction stabilization to alleviate the unstable adversarial optimization caused by the diversity of language references. By coarsely predicting the language-guided sampling process, the optimization process will be stabilized by the designed ResAdv-DDIM sampler, therefore releasing the transferable and robust adversarial capability of multi-step diffusion models. In task modeling, we propose the context-encoded attacking scenario constraint to supplement the missing knowledge from incomplete human instructions. Guidance masking and renderer integration are proposed to regulate the constraints of 2D/3D SemanticAE, activating stronger scenario-adapted attacks. Moreover, in the dimension of generator evaluation, we propose the semantic-abstracted attacking evaluation enhancement by clarifying the evaluation boundary, facilitating the development of more effective SemanticAE generators. Extensive experiments demonstrate the superiority of the transfer attack performance of InSUR. Moreover, we realize the reference-free generation of semantically constrained 3D adversarial examples for the first time.

Problem

• Reducing semantic uncertainty in instruction-based adversarial examples
• Stabilizing adversarial optimization against language reference diversity
• Enhancing transferable and adaptive attacks through uncertainty reduction

Innovation

• ResAdv-DDIM sampler stabilizes adversarial optimization for transferability
• Guidance masking and renderer integration enhance scenario-adapted attacks
• Semantic-abstracted evaluation clarifies boundaries for effective generator development