Enhancing Certifiable Semantic Robustness via Robust Pruning of Deep Neural Networks

2025-09-30

AI Summary

Certified robustness of deep neural networks against semantic transformations (e.g., brightness and contrast changes) suffers from over-parameterization, hindering tight and efficient robustness certification. Method: This paper proposes a structured pruning framework tailored for certifiable robustness. It introduces the Unbiased Smooth Neuron (USN) metric to quantify neuron-level sensitivity to semantic perturbations and guides pruning accordingly. Additionally, it incorporates a Wasserstein-distance-based regularization term into the loss function to enforce concentration of neuron response distributions post-pruning, thereby improving certification tightness. The method jointly leverages layer-wise stability analysis and neuron-wise variance modeling. Contribution/Results: The approach achieves improved model sparsity without compromising verifiable robustness. Evaluated on keypoint detection, it significantly boosts certified coverage and computational efficiency over state-of-the-art baselines.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Deep neural networks have been widely adopted in many vision and robotics applications with visual inputs. It is essential to verify its robustness against semantic transformation perturbations, such as brightness and contrast. However, current certified training and robustness certification methods face the challenge of over-parameterization, which hinders the tightness and scalability due to the over-complicated neural networks. To this end, we first analyze stability and variance of layers and neurons against input perturbation, showing that certifiable robustness can be indicated by a fundamental Unbiased and Smooth Neuron metric (USN). Based on USN, we introduce a novel neural network pruning method that removes neurons with low USN and retains those with high USN, thereby preserving model expressiveness without over-parameterization. To further enhance this pruning process, we propose a new Wasserstein distance loss to ensure that pruned neurons are more concentrated across layers. We validate our approach through extensive experiments on the challenging robust keypoint detection task, which involves realistic brightness and contrast perturbations, demonstrating that our method achieves superior robustness certification performance and efficiency compared to baselines.

Problem

• Addresses over-parameterization in neural network robustness certification
• Enhances certifiable robustness against semantic transformations like brightness
• Improves tightness and scalability of certified training methods

Innovation

• Pruning neurons using Unbiased Smooth Neuron metric
• Applying Wasserstein distance loss for concentration
• Enhancing robustness certification via layer-wise pruning