MeanSparse: Post-Training Robustness Enhancement Through Mean-Centered Feature Sparsification

2024-06-09

arXiv.org

AI Summary

To address the insufficient robustness of CNNs and Transformers against adversarial examples, this paper proposes MeanSparse—a label-free, retraining-free post-training robustness enhancement method. Its core innovation lies in applying mean-centering combined with feature sparsification (under L1/L0 constraints) as a lightweight activation-layer post-processing step, effectively suppressing adversarial perturbation propagation while preserving clean-data accuracy. MeanSparse is the first approach to jointly leverage mean-centering and feature sparsification for post-training robustness improvement and is fully compatible with standard evaluation frameworks such as AutoAttack. On CIFAR-10, CIFAR-100, and ImageNet, it achieves AutoAttack accuracies of 75.28%, 44.78%, and 62.12%, respectively—setting new state-of-the-art records on RobustBench and significantly improving the robustness–accuracy trade-off.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Privacy-enhancing technologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Measurement, analysis, and circumvention of Web censorship

Abstract

We present a simple yet effective method to improve the robustness of both Convolutional and attention-based Neural Networks against adversarial examples by post-processing an adversarially trained model. Our technique, MeanSparse, cascades the activation functions of a trained model with novel operators that sparsify mean-centered feature vectors. This is equivalent to reducing feature variations around the mean, and we show that such reduced variations merely affect the model's utility, yet they strongly attenuate the adversarial perturbations and decrease the attacker's success rate. Our experiments show that, when applied to the top models in the RobustBench leaderboard, MeanSparse achieves a new robustness record of 75.28% (from 73.71%), 44.78% (from 42.67%) and 62.12% (from 59.56%) on CIFAR-10, CIFAR-100 and ImageNet, respectively, in terms of AutoAttack accuracy. Code is available at https://github.com/SPIN-UMass/MeanSparse

Problem

• Improving neural network robustness against adversarial examples
• Reducing feature variations to attenuate adversarial perturbations
• Enhancing post-trained model security with mean-centered sparsification

Innovation

• Post-training method enhances model robustness
• Sparsifies mean-centered feature vectors in networks
• Reduces feature variations to attenuate adversarial perturbations