S2AP: Score-space Sharpness Minimization for Adversarial Pruning

2025-10-21

AI Summary

Existing adversarial pruning methods optimize importance scores to minimize robust loss but often converge to sharp local minima, leading to unstable mask selection and degraded robustness. To address this, we propose Score-space Sharpness-aware Adversarial Pruning (S2AP), the first method to explicitly define and minimize *score-space sharpness*—i.e., the worst-case robust loss under perturbations in the importance score space—thereby stabilizing mask search. S2AP jointly integrates importance-based pruning, adversarial training, and sharpness-aware optimization without altering network architecture or training paradigms. Extensive experiments across multiple datasets, models, and sparsity levels demonstrate that S2AP significantly reduces score-space sharpness, consistently improves post-pruning robust accuracy, and outperforms state-of-the-art adversarial pruning approaches.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Adversarial Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Adversarial pruning methods have emerged as a powerful tool for compressing neural networks while preserving robustness against adversarial attacks. These methods typically follow a three-step pipeline: (i) pretrain a robust model, (ii) select a binary mask for weight pruning, and (iii) finetune the pruned model. To select the binary mask, these methods minimize a robust loss by assigning an importance score to each weight, and then keep the weights with the highest scores. However, this score-space optimization can lead to sharp local minima in the robust loss landscape and, in turn, to an unstable mask selection, reducing the robustness of adversarial pruning methods. To overcome this issue, we propose a novel plug-in method for adversarial pruning, termed Score-space Sharpness-aware Adversarial Pruning (S2AP). Through our method, we introduce the concept of score-space sharpness minimization, which operates during the mask search by perturbing importance scores and minimizing the corresponding robust loss. Extensive experiments across various datasets, models, and sparsity levels demonstrate that S2AP effectively minimizes sharpness in score space, stabilizing the mask selection, and ultimately improving the robustness of adversarial pruning methods.

Problem

• Minimizes sharp local minima in robust loss landscape
• Stabilizes mask selection during adversarial pruning process
• Improves robustness preservation in neural network compression

Innovation

• Minimizes score-space sharpness for pruning
• Perturbs importance scores during mask search
• Stabilizes mask selection to improve robustness