When Does Pruning Benefit Vision Representations?

2025-07-02

AI Summary

This study systematically investigates the tripartite impact of model pruning on visual representations—specifically interpretability, unsupervised object discovery capability, and alignment with human perception. Method: We conduct experiments across multiple mainstream architectures of varying parameter scales and sparsity levels, integrating feature attribution analysis, representation structure metrics, and human behavioral benchmarking. Contribution/Results: We identify a “sweet spot” in sparsity: moderate (non-extreme) pruning simultaneously enhances attribution interpretability, structural coherence in object discovery, and alignment with human attention patterns—effects modulated by architectural complexity. In contrast, excessive pruning degrades performance. Critically, we reveal for the first time a non-monotonic coupling among sparsity level, architectural scale, and representation quality. These findings provide novel empirical evidence and design principles for interpretable AI and biologically plausible representation learning.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Representation Learning
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Pruning is widely used to reduce the complexity of deep learning models, but its effects on interpretability and representation learning remain poorly understood. This paper investigates how pruning influences vision models across three key dimensions: (i) interpretability, (ii) unsupervised object discovery, and (iii) alignment with human perception. We first analyze different vision network architectures to examine how varying sparsity levels affect feature attribution interpretability methods. Additionally, we explore whether pruning promotes more succinct and structured representations, potentially improving unsupervised object discovery by discarding redundant information while preserving essential features. Finally, we assess whether pruning enhances the alignment between model representations and human perception, investigating whether sparser models focus on more discriminative features similarly to humans. Our findings also reveal the presence of sweet spots, where sparse models exhibit higher interpretability, downstream generalization and human alignment. However, these spots highly depend on the network architectures and their size in terms of trainable parameters. Our results suggest a complex interplay between these three dimensions, highlighting the importance of investigating when and how pruning benefits vision representations.

Problem

• How pruning affects vision model interpretability and feature attribution
• Whether pruning improves unsupervised object discovery via structured representations
• If pruning enhances model-human perception alignment by focusing on discriminative features

Innovation

• Analyzing pruning effects on interpretability methods
• Exploring pruning for structured unsupervised object discovery
• Assessing pruning alignment with human perception