Lower Bounds on Adversarial Robustness for Multiclass Classification with General Loss Functions

2025-10-02

AI Summary

This work addresses the problem of deriving tight lower bounds on adversarial robustness for general multiclass loss functions—including cross-entropy, power-loss, and quadratic loss—overcoming the limitation of prior theories restricted to 0–1 loss. Methodologically, we propose a learner-agnostic dual characterization and barycentric reformulation of the robust risk minimization problem, establishing—for the first time—its intrinsic connection to α-fair binning and generalized barycenter problems. Leveraging an α-divergence framework regularized by KL divergence and Tsallis entropy, we integrate dual optimization with barycentric reconstruction to derive computationally tractable, tight lower bounds. Empirically, our approach significantly improves bound tightness under standard losses such as cross-entropy, yielding a more general and structurally insightful theoretical foundation for adversarial robustness in multiclass classification.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

We consider adversarially robust classification in a multiclass setting under arbitrary loss functions and derive dual and barycentric reformulations of the corresponding learner-agnostic robust risk minimization problem. We provide explicit characterizations for important cases such as the cross-entropy loss, loss functions with a power form, and the quadratic loss, extending in this way available results for the 0-1 loss. These reformulations enable efficient computation of sharp lower bounds for adversarial risks and facilitate the design of robust classifiers beyond the 0-1 loss setting. Our paper uncovers interesting connections between adversarial robustness, $α$-fair packing problems, and generalized barycenter problems for arbitrary positive measures where Kullback-Leibler and Tsallis entropies are used as penalties. Our theoretical results are accompanied with illustrative numerical experiments where we obtain tighter lower bounds for adversarial risks with the cross-entropy loss function.

Problem

• Derives dual formulations for multiclass adversarial risk minimization
• Provides explicit characterizations for cross-entropy and power loss functions
• Establishes connections between adversarial robustness and barycenter problems

Innovation

• Dual reformulations for multiclass robust risk
• Barycentric methods with entropy-based penalties
• Sharp lower bounds for cross-entropy adversarial risks