Model Compression using Progressive Channel Pruning

2025-07-07

AI Summary

To address the accuracy-efficiency trade-off in convolutional neural network (CNN) model compression, this paper proposes Progressive Channel Pruning (PCP), a novel iterative framework. PCP employs a “trial-selection-pruning” three-stage mechanism: it evaluates pruning errors on a validation set and—using a greedy strategy—automatically identifies and sparsifies only the least impactful layers, thereby avoiding instability inherent in global optimization. The method is compatible with mainstream channel pruning algorithms and, for the first time, extends to domain-adversarial networks and other transfer learning models. Additionally, PCP introduces joint optimization over labeled data and pseudo-labels to enhance adaptability in unsupervised and semi-supervised settings. Evaluated on two benchmark datasets, PCP consistently outperforms state-of-the-art methods: under equal compression ratios, it reduces average accuracy degradation by 35% and accelerates inference by 2.1×, achieving an effective balance between high accuracy retention and efficient compression.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Learning & Optimization for CV
• Constraint Satisfaction and Optimization: Constraint Programming

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Security and Privacy: Large-scale security measurements

Abstract

In this work, we propose a simple but effective channel pruning framework called Progressive Channel Pruning (PCP) to accelerate Convolutional Neural Networks (CNNs). In contrast to the existing channel pruning methods that prune channels only once per layer in a layer-by-layer fashion, our new progressive framework iteratively prunes a small number of channels from several selected layers, which consists of a three-step attempting-selecting-pruning pipeline in each iteration. In the attempting step, we attempt to prune a pre-defined number of channels from one layer by using any existing channel pruning methods and estimate the accuracy drop for this layer based on the labelled samples in the validation set. In the selecting step, based on the estimated accuracy drops for all layers, we propose a greedy strategy to automatically select a set of layers that will lead to less overall accuracy drop after pruning these layers. In the pruning step, we prune a small number of channels from these selected layers. We further extend our PCP framework to prune channels for the deep transfer learning methods like Domain Adversarial Neural Network (DANN), in which we effectively reduce the data distribution mismatch in the channel pruning process by using both labelled samples from the source domain and pseudo-labelled samples from the target domain. Our comprehensive experiments on two benchmark datasets demonstrate that our PCP framework outperforms the existing channel pruning approaches under both supervised learning and transfer learning settings.

Problem

• Accelerate CNNs via iterative channel pruning
• Reduce accuracy drop with greedy layer selection
• Prune channels in transfer learning effectively

Innovation

• Iterative pruning of selected CNN channels
• Greedy layer selection for minimal accuracy drop
• Domain adaptation with source and pseudo-labeled data