Adversarial robustness through Lipschitz-Guided Stochastic Depth in Neural Networks

2025-09-12

AI Summary

Deep neural networks and Vision Transformers (ViTs) remain vulnerable to adversarial attacks, while existing defenses suffer from high computational overhead or lack theoretical guarantees. To address this, we propose Lipschitz-guided stochastic depth: a depth-dependent DropPath strategy that dynamically controls the effective Lipschitz constant of the network, enabling structured regularization during training. Specifically, on ViT-Tiny with an incrementally increasing drop probability schedule, our method achieves significant gains in adversarial robustness against FGSM, PGD-20, and AutoAttack on CIFAR-10, while preserving clean accuracy close to the baseline and reducing FLOPs—outperforming both standard and linear DropPath baselines. Our key contribution is the first integration of Lipschitz continuity constraints with depth-aware stochastic depth, jointly optimizing robustness, accuracy, and efficiency.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Deep neural networks and Vision Transformers achieve state-of-the-art performance in computer vision but are highly vulnerable to adversarial perturbations. Standard defenses often incur high computational cost or lack formal guarantees. We propose a Lipschitz-guided stochastic depth (DropPath) method, where drop probabilities increase with depth to control the effective Lipschitz constant of the network. This approach regularizes deeper layers, improving robustness while preserving clean accuracy and reducing computation. Experiments on CIFAR-10 with ViT-Tiny show that our custom depth-dependent schedule maintains near-baseline clean accuracy, enhances robustness under FGSM, PGD-20, and AutoAttack, and significantly reduces FLOPs compared to baseline and linear DropPath schedules.

Problem

• Improving adversarial robustness in neural networks
• Reducing computational costs of defense methods
• Maintaining clean accuracy while enhancing security

Innovation

• Lipschitz-guided stochastic depth method
• DropPath probabilities increase with depth
• Controls network's effective Lipschitz constant