Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization

2025-05-16

AI Summary

To address the accuracy degradation in post-training quantization of neural networks—caused by coupled and accumulated quantization errors across weights, activations, and preceding-layer outputs—this paper proposes Qronos, an iterative quantization algorithm that jointly corrects multi-source errors. Its core contributions are threefold: (i) the first explicit cross-layer modeling and joint correction of quantization errors in weights, activations, and KV caches; (ii) an efficient least-squares optimization framework based on Cholesky decomposition, enhanced by Hadamard transformation to improve numerical conditioning; and (iii) a weight-activation scaling balance mechanism to boost robustness. Evaluated on the Llama3 family, Qronos achieves state-of-the-art performance under joint W4A4KV4 quantization, significantly outperforming adaptive rounding baselines. This demonstrates both theoretical rigor and practical efficacy in mitigating error propagation across transformer layers.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We introduce Qronos -- a new state-of-the-art post-training quantization algorithm that sequentially rounds and updates neural network weights. Qronos not only explicitly corrects errors due to both weight and activation quantization, but also errors resulting from quantizing previous layers. Our iterative algorithm is based on an interpretable and disciplined optimization framework that subsumes and surpasses existing data-driven approaches. At each step, Qronos alternates between error correction and diffusion via optimal update rules. Importantly, we prove that Qronos admits an efficient implementation that uses the Cholesky decomposition for solving least-squares problems. We also demonstrate that Qronos is compatible with existing transformation techniques such as Hadamard-based incoherence processing and weight-activation scaling equalization, among others. We evaluate Qronos using recent autoregressive language generation models in the Llama3 family; Qronos consistently outperforms previous state-of-the-art adaptive rounding methods when quantizing the weights, activations, and/or KV caches.

Problem

• Corrects errors from weight and activation quantization
• Addresses errors from quantizing previous layers
• Improves post-training quantization for neural networks

Innovation

• Sequentially rounds and updates neural network weights
• Corrects errors from weight and activation quantization
• Uses Cholesky decomposition for efficient least-squares solving