LinDeps: A Fine-tuning Free Post-Pruning Method to Remove Layer-Wise Linear Dependencies with Guaranteed Performance Preservation

2025-07-29

AI Summary

Existing CNN pruning methods often overlook intra-layer linear dependencies among feature maps, leading to insufficient removal of redundant filters and constrained compression ratios. To address this, we propose LinDeps, a fine-tuning-free post-training pruning method. Its core innovation is the first application of pivoted QR decomposition to quantify linear dependencies among filters within a layer, enabling precise identification of redundant filters. Furthermore, LinDeps introduces a feature map reconstruction mechanism coupled with compensation of downstream convolutional kernels to restore signal flow and preserve model expressiveness after filter removal. The method is modular and can be seamlessly integrated into existing pruning pipelines. Experiments on CIFAR-10 and ImageNet demonstrate that LinDeps significantly improves compression ratios (up to 2.3×) and inference speed on VGG and ResNet architectures, outperforming state-of-the-art methods—even without any retraining.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Other Foundations of Computer Vision
• Search and Optimization: Non-convex Optimization

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Convolutional Neural Networks (CNN) are widely used in many computer vision tasks. Yet, their increasing size and complexity pose significant challenges for efficient deployment on resource-constrained platforms. Hence, network pruning has emerged as an effective way of reducing the size and computational requirements of neural networks by removing redundant or unimportant parameters. However, a fundamental challenge with pruning consists in optimally removing redundancies without degrading performance. Most existing pruning techniques overlook structural dependencies across feature maps within a layer, resulting in suboptimal pruning decisions. In this work, we introduce LinDeps, a novel post-pruning method, i.e., a pruning method that can be applied on top of any pruning technique, which systematically identifies and removes redundant filters via linear dependency analysis. Particularly, LinDeps applies pivoted QR decomposition to feature maps to detect and prune linearly dependent filters. Then, a novel signal recovery mechanism adjusts the next layer's kernels to preserve compatibility and performance without requiring any fine-tuning. Our experiments on CIFAR-10 and ImageNet with VGG and ResNet backbones demonstrate that LinDeps improves compression rates of existing pruning techniques while preserving performances, leading to a new state of the art in CNN pruning. We also benchmark LinDeps in low-resource setups where no retraining can be performed, which shows significant pruning improvements and inference speedups over a state-of-the-art method. LinDeps therefore constitutes an essential add-on for any current or future pruning technique.

Problem

• Removes redundant CNN filters via linear dependency analysis
• Preserves performance without requiring fine-tuning
• Improves compression rates in resource-constrained setups

Innovation

• Uses pivoted QR decomposition for linear dependency analysis
• Applies signal recovery mechanism without fine-tuning
• Improves compression rates while preserving performance