Budget-Aware Adaptive Adversarial Patches for Black-Box Object Detection

2026-06-16

AI Summary

Existing black-box adversarial attacks on object detectors struggle to jointly optimize patch location, texture, and size under strict query budgets and lack clear evaluation of visual footprint versus actual suppression efficacy. This work proposes a query-efficient, budget-adaptive attack that integrates a lightweight contextual Thompson sampling-based localizer with an NES-style pixel update strategy. The method adaptively expands patch size only when attack progress stalls and incorporates printability constraints to balance perturbation strength and visual perceptibility. It is the first to achieve joint adaptive optimization of location, texture, and size in a black-box setting while explicitly distinguishing between Expectation Over Transformation (EOT) robustness and real-world suppression performance. Experiments demonstrate significant improvements over fixed-size and heuristic baselines on YOLOv5, Faster R-CNN, and YOLOS, with print-and-shoot tests confirming strong physical transferability across objects and viewpoints.

Abstract

Adversarial patches pose a practical threat to modern object detectors. Prior work shows vulnerability, but three gaps limit actionable insight: (i) few \emph{score-based black-box} attacks \emph{jointly} optimize patch \emph{location, texture, and size} under tight query budgets; (ii) success is rarely tied to the patch's \emph{visual footprint}; and (iii) evaluations often conflate EOT robustness with plain-view suppression. We present \method{}, a query-efficient, budget-adaptive black-box attack that couples a lightweight \emph{Contextual Thompson-Sampling} placer with NES-style pixel updates, growing the patch only when progress stalls. Reporting is anchored by a \emph{strict plain-image} suppression test; EOT is audited but never used as a substitute for success, and optional appearance/printability weights expose strength--visibility trade-offs. Across YOLOv5, Faster R-CNN, and YOLOS, \method{} achieves strong suppression on CNN-based detectors and substantial suppression on the transformer-based detector, using compact patches and exposing clear query--footprint trade-offs relative to fixed-size and heuristic baselines. A print--capture pilot further shows transfer across unseen physical objects and viewpoints.

Problem

• adversarial patches
• black-box attack
• query budget
• visual footprint
• object detection

Innovation

• black-box attack
• adversarial patch
• query efficiency
• adaptive patch sizing
• object detection