Systematic Characterization of Minimal Deep Learning Architectures: A Unified Analysis of Convergence, Pruning, and Quantization

2026-01-25

AI Summary

This study systematically investigates the stability and efficiency of deep learning architectures under pruning and low-precision quantization constraints in image classification tasks, aiming to identify minimally viable models. By unifying the modeling of convergence behavior, pruning sensitivity, and quantization robustness across DNNs, CNNs, and Vision Transformers (ViTs), the work reveals for the first time the intrinsic coupling among these properties and quantifies the minimal learnable parameter count. Through large-scale architecture search and extensive experiments across multiple datasets, the authors demonstrate that model performance is primarily governed by parameter count rather than architectural type, that up to 60% of parameters in deep networks can be pruned without significant degradation, and that smaller models and more complex datasets exhibit heightened sensitivity to quantization. These findings provide both theoretical grounding and practical guidance for efficient model deployment.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Deep learning networks excel at classification, yet identifying minimal architectures that reliably solve a task remains challenging. We present a computational methodology for systematically exploring and analyzing the relationships among convergence, pruning, and quantization. The workflow first performs a structured design sweep across a large set of architectures, then evaluates convergence behavior, pruning sensitivity, and quantization robustness on representative models. Focusing on well-known image classification of increasing complexity, and across Deep Neural Networks, Convolutional Neural Networks, and Vision Transformers, our initial results show that, despite architectural diversity, performance is largely invariant and learning dynamics consistently exhibit three regimes: unstable, learning, and overfitting. We further characterize the minimal learnable parameters required for stable learning, uncover distinct convergence and pruning phases, and quantify the effect of reduced numeric precision on trainable parameters. Aligning with intuition, the results confirm that deeper architectures are more resilient to pruning than shallower ones, with parameter redundancy as high as 60%, and quantization impacts models with fewer learnable parameters more severely and has a larger effect on harder image datasets. These findings provide actionable guidance for selecting compact, stable models under pruning and low-precision constraints in image classification.

Problem

• minimal architectures
• convergence
• pruning
• quantization
• image classification

Innovation

• systematic characterization
• minimal architectures
• convergence-pruning-quantization analysis
• learning dynamics
• parameter redundancy