Advanced deep architecture pruning using single filter performance

2025-01-22

AI Summary

To address the high computational cost and low energy efficiency caused by parameter redundancy in deep neural networks, this paper proposes a structured pruning method based on single-filter performance evaluation. The core contribution is twofold: (1) it extends the single-filter performance quantification mechanism into a scalable cluster-level pruning strategy—termed Adaptive Filter Clustering and Compression (AFCC)—unified across both convolutional and fully connected layers; and (2) it introduces an inter-layer structural awareness mechanism that reveals the correlation between local filter contributions and global model accuracy. Evaluated on CIFAR-100 with VGG-11 and EfficientNet-B0, the method achieves near-zero accuracy degradation under identical pruning ratios, while significantly reducing parameter count and FLOPs. It consistently outperforms state-of-the-art pruning approaches in terms of accuracy–efficiency trade-off.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Pruning the parameters and structure of neural networks reduces the computational complexity, energy consumption, and latency during inference. Recently, a novel underlying mechanism for successful deep learning (DL) was presented based on a method that quantitatively measures the single filter performance in each layer of a DL architecture, and a new comprehensive mechanism of how deep learning works was presented. Herein, we demonstrate how this understanding paves the path to highly dilute the convolutional layers of deep architectures without affecting their overall accuracy using applied filter cluster connections (AFCC). AFCC is exemplified on VGG-11 and EfficientNet-B0 architectures trained on CIFAR-100, and its high pruning outperforms other techniques using the same pruning magnitude. Additionally, this technique is broadened to single nodal performance and highly pruning of fully connected layers, suggesting a possible implementation to considerably reduce the complexity of over-parameterized AI tasks.

Problem

• Optimization
• Deep Learning Models
• Computational Efficiency

Innovation

• AFCC
• Complexity Reduction
• Neural Pruning