D2R: dual regularization loss with collaborative adversarial generation for model robustness

2025-06-08

AI Summary

To address the insufficient adversarial robustness of deep neural networks, this paper proposes the Dual-Regularized Loss (D2R Loss) and the Collaborative Adversarial Generation (CAG) framework. Methodologically: (1) we design a distribution-aware, two-stage regularized loss that jointly optimizes the model on both clean and adversarial data distributions; (2) we introduce the first gradient-level collaborative adversarial generation mechanism, enabling differentiable and dynamic co-construction of adversarial examples between a guiding model and a target model. Evaluated on WideResNet34-10 and PreActResNet18 across CIFAR-10/100 and Tiny ImageNet, our approach achieves average robust accuracy improvements of 3.2–5.7 percentage points over state-of-the-art methods, with significantly enhanced generalization against diverse adversarial attacks. The core contributions are the novel *distribution-collaborative regularization* paradigm and the first *gradient-driven, model-collaborative adversarial generation* framework.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

The robustness of Deep Neural Network models is crucial for defending models against adversarial attacks. Recent defense methods have employed collaborative learning frameworks to enhance model robustness. Two key limitations of existing methods are (i) insufficient guidance of the target model via loss functions and (ii) non-collaborative adversarial generation. We, therefore, propose a dual regularization loss (D2R Loss) method and a collaborative adversarial generation (CAG) strategy for adversarial training. D2R loss includes two optimization steps. The adversarial distribution and clean distribution optimizations enhance the target model's robustness by leveraging the strengths of different loss functions obtained via a suitable function space exploration to focus more precisely on the target model's distribution. CAG generates adversarial samples using a gradient-based collaboration between guidance and target models. We conducted extensive experiments on three benchmark databases, including CIFAR-10, CIFAR-100, Tiny ImageNet, and two popular target models, WideResNet34-10 and PreActResNet18. Our results show that D2R loss with CAG produces highly robust models.

Problem

• Enhancing DNN robustness against adversarial attacks
• Addressing insufficient loss function guidance in defenses
• Improving adversarial sample generation via collaborative strategies

Innovation

• Dual regularization loss enhances robustness
• Collaborative adversarial generation strategy
• Gradient-based adversarial sample generation