ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers

2026-06-20

AI Summary

This work addresses the performance degradation of static post-training quantization (PTQ) for Vision Transformers, which arises from input-dependent variations in activation distributions. To overcome this limitation, the authors propose ScalePredictor, a dynamic quantization framework that achieves instance-aware quantization without on-the-fly calibration—a first in the field. The core insight is an implicit correlation between shallow-layer activation ranges and optimal quantization scales for deeper layers, enabling the prediction of deep-layer scales from early activations. ScalePredictor integrates an efficient range extraction mechanism with a Taylor-expansion-based polynomial scale projection module. Evaluated on ImageNet, the method substantially outperforms existing PTQ approaches, achieving a superior accuracy-efficiency trade-off with negligible computational overhead.

Abstract

Vision Transformers have achieved remarkable success in many fields, yet their deployment on edge devices remains challenging due to their substantial computational demands. Post-Training Quantization (PTQ) offers an attractive solution by compressing models using a small calibration set with minimal training overhead. However, most existing PTQ works adopt a static quantization paradigm that is uniformly applied to all instances. Given the substantial diversity of natural images, the activation distributions vary significantly across samples, making these methods inherently suboptimal. In this paper, we propose ScalePredictor, a dynamic quantization framework for accurate and efficient quantization scale learning of ViTs. We first reveal a hidden correlation between the distribution range of shallow-layer activations and the optimal scales of deeper layers. Based on this, we develop a scale learning mechanism that integrates an efficient range extraction approach to capture robust range statistics at the shallow stage, which are then fed into a Taylor-motivated polynomial scale projection module to generate all quantization scales simultaneously. With the efficiency of polynomial approximation, ScalePredictor introduces insignificant computational overhead while avoiding costly just-in-time calibration. Extensive experiments on ImageNet demonstrate that ScalePredictor consistently outperforms prior PTQ methods, achieving a more favorable accuracy-efficiency trade-off. Code and additional results are shown in the supplementary materials.

Problem

• Post-Training Quantization
• Vision Transformers
• Instance-aware Quantization
• Activation Distribution
• Quantization Scale

Innovation

• dynamic quantization
• vision transformers
• post-training quantization
• scale prediction
• polynomial approximation