StatQAT: Statistical Quantizer Optimization for Deep Networks

2026-05-17

AI Summary

This work addresses the quantization error in deep neural networks caused by diverse data distributions by proposing a unified framework grounded in statistical error analysis, which for the first time tightly integrates quantizer design with the characteristics of data distributions. The framework encompasses an iterative optimization-based quantizer applicable to arbitrary distributions and an analytical quantizer tailored for near-Gaussian weight distributions, supporting both integer and floating-point formats and seamlessly integrating with quantization-aware training (QAT). Experimental results demonstrate that the proposed approach significantly improves accuracy and training stability of low-bit models across various architectures and datasets, outperforming existing uniform and floating-point quantization methods.

Abstract

Quantization is essential for reducing the computational cost and memory usage of deep neural networks, enabling efficient inference on low-precision hardware. Despite the growing adoption of uniform and floating-point quantization schemes, selecting optimal quantization parameters remains a key challenge, particularly for diverse data distributions encountered during training and inference. This work presents a novel statistical error analysis framework for uniform and floating-point quantization, providing theoretical insight into error behavior across quantization configurations. Building on this analysis, we propose iterative quantizers designed for arbitrary data distributions and analytic quantizers tailored for Gaussian-like weight distributions. These methods enable efficient, low-error quantization suitable for both activations and weights. We incorporate our quantizers into quantization-aware training and evaluate them across integer and floating-point formats. Experiments demonstrate improved accuracy and stability, highlighting the effectiveness of our approach for training low-precision neural networks.

Problem

• quantization
• deep neural networks
• quantization parameters
• data distributions
• low-precision inference

Innovation

• statistical error analysis
• iterative quantizer
• analytic quantizer
• quantization-aware training
• low-precision neural networks