Technical Report on the CVPR 2026@AdvML Workshop Challenge

2026-07-13

AI Summary

This work addresses the vulnerability of Vision-Language Agents (VLAs) in autonomous driving under multimodal adversarial attacks by organizing a challenge centered on DriveLM-style multi-view visual question answering. Participants are tasked with generating high-fidelity adversarial images and text perturbations with minimal textual distortion to mislead models into producing answers that deviate from reference responses, while evaluating transferability under both white-box and black-box settings. The study presents the first systematic assessment of multi-view multimodal attacks, introducing novel techniques such as QA-graph-guided budget allocation, feature-space optimization, and suffix-constrained textual perturbations. Key findings include the dominance of image-side attacks, the efficacy of scene-level optimization, and the susceptibility of layout-sensitive content, thereby establishing a benchmark for robustness evaluation and defense development in VLAs.

Abstract

Vision-language agents (VLAs) are increasingly used to interpret complex driving scenes and support safety-critical reasoning. This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs. Built on DriveLM-style multi-view visual question answering, the challenge represents each scene with six synchronized camera images and a structured collection of driving-related question-answer pairs. Participants generate adversarial images and suffix-only textual perturbations that induce model responses to deviate from reference answers while preserving image fidelity and limiting textual cost. The competition comprises two phases, with Phase II adding a hidden black-box model to assess transferability. We describe the task design, submission rules, evaluation protocol, and leaderboard results, and then examine five leading submissions for which technical reports were available. Across these reports, several recurring patterns emerge: image-side attacks are favored by the suffix penalty; scene-level, multi-view optimization is more effective than treating views in isolation; QA types and graph structure provide useful priors for allocating attack budget; feature-space objectives can improve black-box transfer; and typographic content embedded in camera images exposes a persistent vulnerability in driving VLAs. These findings provide a practical reference for future robustness evaluation and defense design in multimodal autonomous-driving systems.

Problem

• vision-language agents
• adversarial attacks
• autonomous driving
• multimodal robustness
• visual question answering

Innovation

• multimodal adversarial attack
• vision-language agents
• multi-view optimization
• black-box transferability
• typographic vulnerability