Rewriting the Budget: A General Framework for Black-Box Attacks Under Cost Asymmetry

2025-06-07

AI Summary

This paper addresses an overlooked practical challenge in decision-based black-box adversarial attacks: asymmetric query costs—e.g., certain outputs trigger expensive human review. We propose the first general optimization framework for this setting. Methodologically, we introduce Asymmetric Search (AS) and Asymmetric Gradient Estimation (AGREST), jointly optimizing both query-type distribution and search strategy to minimize weighted total cost under perturbation magnitude constraints. Our framework integrates conservative binary search, reweighted sampling for gradient estimation, and cost-aware scheduling, seamlessly extending state-of-the-art methods including Boundary Attack and QEBA. We provide theoretical convergence guarantees. Experiments on ImageNet and CIFAR demonstrate up to 40% reduction in total query cost compared to prior work, alongside smaller adversarial perturbations, establishing significant improvements over existing approaches.

Technology Category

• Search and Optimization: Adversarial Search
• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Traditional decision-based black-box adversarial attacks on image classifiers aim to generate adversarial examples by slightly modifying input images while keeping the number of queries low, where each query involves sending an input to the model and observing its output. Most existing methods assume that all queries have equal cost. However, in practice, queries may incur asymmetric costs; for example, in content moderation systems, certain output classes may trigger additional review, enforcement, or penalties, making them more costly than others. While prior work has considered such asymmetric cost settings, effective algorithms for this scenario remain underdeveloped. In this paper, we propose a general framework for decision-based attacks under asymmetric query costs, which we refer to as asymmetric black-box attacks. We modify two core components of existing attacks: the search strategy and the gradient estimation process. Specifically, we propose Asymmetric Search (AS), a more conservative variant of binary search that reduces reliance on high-cost queries, and Asymmetric Gradient Estimation (AGREST), which shifts the sampling distribution to favor low-cost queries. We design efficient algorithms that minimize total attack cost by balancing different query types, in contrast to earlier methods such as stealthy attacks that focus only on limiting expensive (high-cost) queries. Our method can be integrated into a range of existing black-box attacks with minimal changes. We perform both theoretical analysis and empirical evaluation on standard image classification benchmarks. Across various cost regimes, our method consistently achieves lower total query cost and smaller perturbations than existing approaches, with improvements of up to 40% in some settings.

Problem

• Address asymmetric query costs in black-box attacks
• Optimize search strategy to reduce high-cost queries
• Balance query types to minimize total attack cost

Innovation

• Asymmetric Search reduces high-cost query reliance
• Asymmetric Gradient Estimation favors low-cost queries
• Balances query types to minimize total attack cost