ARMOR: Agentic Reasoning for Methods Orchestration and Reparameterization for Robust Adversarial Attacks

2026-01-26

AI Summary

This work proposes an adaptive adversarial attack framework based on collaborative agent reasoning, addressing the limitations of existing methods that rely on static strategy ensembles and lack semantic awareness or dynamic adaptability. The framework leverages a vision-language model (VLM) to identify semantic vulnerabilities and orchestrates three attack primitives—CW, JSMA, and STA—to generate and fuse perturbations on a shared “mixing board.” A large language model (LLM) enables real-time, closed-loop reparameterization of the attack strategy. By synergistically integrating VLM and LLM for the first time in adversarial attacks, this approach transcends conventional static ensemble paradigms, significantly improving cross-model transferability and attack success rates on standard benchmarks. It delivers fused attacks against black-box targets and adaptively selects either optimal or fused strategies for white-box targets, achieving a balanced trade-off between effectiveness and robustness.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Existing automated attack suites operate as static ensembles with fixed sequences, lacking strategic adaptation and semantic awareness. This paper introduces the Agentic Reasoning for Methods Orchestration and Reparameterization (ARMOR) framework to address these limitations. ARMOR orchestrates three canonical adversarial primitives, Carlini-Wagner (CW), Jacobian-based Saliency Map Attack (JSMA), and Spatially Transformed Attacks (STA) via Vision Language Models (VLM)-guided agents that collaboratively generate and synthesize perturbations through a shared ``Mixing Desk". Large Language Models (LLMs) adaptively tune and reparameterize parallel attack agents in a real-time, closed-loop system that exploits image-specific semantic vulnerabilities. On standard benchmarks, ARMOR achieves improved cross-architecture transfer and reliably fools both settings, delivering a blended output for blind targets and selecting the best attack or blended attacks for white-box targets using a confidence-and-SSIM score.

Problem

• adversarial attacks
• static ensembles
• strategic adaptation
• semantic awareness
• cross-architecture transfer

Innovation

• Agentic Reasoning
• Methods Orchestration
• Reparameterization
• Vision Language Models
• Adversarial Attacks