🤖 AI Summary
To address deployment constraints on resource-limited devices, this paper proposes a fine-tuning-free post-training quantization (PTQ) method for non-uniform weight quantization. To overcome the accuracy bottlenecks of conventional uniform or channel-wise quantization, we introduce, for the first time in PTQ, a theoretically grounded noise-minimization mechanism that jointly optimizes clipping thresholds and scaling factors. Our approach leverages gradient-free statistical modeling, hierarchical non-uniform clustering, and convex optimization to achieve distribution-adaptive quantization. Evaluated on real-world datasets, our method achieves 4–8× model compression, 3.2× inference speedup, and less than 0.3% Top-1 accuracy degradation—substantially outperforming state-of-the-art PTQ baselines. Key contributions include: (i) a theoretically guaranteed optimal non-uniform quantization design; (ii) the first provably noise-minimizing PTQ framework; and (iii) a highly efficient, entirely fine-tuning-free implementation.
📝 Abstract
Despite the success of CNN models on a variety of Image classification and segmentation tasks, their extensive computational and storage demands pose considerable challenges for real-world deployment on resource constrained devices. Quantization is one technique that aims to alleviate these large storage requirements and speed up the inference process by reducing the precision of model parameters to lower-bit representations. In this paper, we introduce a novel post-training quantization method for model weights. Our method finds optimal clipping thresholds and scaling factors along with mathematical guarantees that our method minimizes quantization noise. Empirical results on Real World Datasets demonstrate that our quantization scheme significantly reduces model size and computational requirements while preserving model accuracy.