Quadratic Upper Bound for Boosting Robustness

2026-01-20

International Conference on Machine Learning

AI Summary

This work addresses the robustness degradation commonly observed in Fast Adversarial Training (FAT), which stems from insufficient exploration of adversarial examples. To mitigate this issue, the authors propose a novel optimization approach based on a Quadratic Upper Bound (QUB) of the adversarial training loss function. By constructing and integrating this upper bound into existing FAT frameworks, the method effectively smooths the loss landscape, thereby alleviating robustness deterioration. Extensive experiments demonstrate that the proposed technique significantly enhances model robustness across multiple standard benchmarks, underscoring the critical role of loss function smoothing in improving the performance of adversarial training.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Adversarial Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Fast adversarial training (FAT) aims to enhance the robustness of models against adversarial attacks with reduced training time, however, FAT often suffers from compromised robustness due to insufficient exploration of adversarial space. In this paper, we develop a loss function to mitigate the problem of degraded robustness under FAT. Specifically, we derive a quadratic upper bound (QUB) on the adversarial training (AT) loss function and propose to utilize the bound with existing FAT methods. Our experimental results show that applying QUB loss to the existing methods yields significant improvement of robustness. Furthermore, using various metrics, we demonstrate that this improvement is likely to result from the smoothened loss landscape of the resulting model.

Problem

• fast adversarial training
• robustness degradation
• adversarial space exploration
• loss landscape

Innovation

• Quadratic Upper Bound
• Fast Adversarial Training
• Robustness
• Loss Landscape Smoothing
• Adversarial Training