$\text{Log}_\text{b}$Quant: Quantizing Language Models in Logarithmic Space

2026-07-01

AI Summary

Traditional uniform quantization struggles to effectively represent low-frequency, high-magnitude weights in language models, leading to significant accuracy degradation. This work proposes a tensor-wise asymmetric logarithmic quantization method that introduces an adjustable base mechanism to flexibly adapt to the intrinsic parameter distribution, thereby overcoming the performance limitations of 4-bit linear quantization. The proposed approach consistently outperforms existing 4-bit linear quantization schemes across multiple benchmarks, achieving substantial memory reduction while enabling modest inference acceleration—making it well-suited for deployment on consumer-grade GPUs.

Abstract

Quantization has become an invaluable tool to reduce memory requirements and inference speed of modern language models, in particular to make them available for consumer setups and edge devices. While previous work has primarily focused on uniform quantization codebooks, such approaches are prone to suboptimal representations due to low-frequency high-magnitude weights. We introduce Log$_\text{b}$Quant, a novel logarithmic quantization approach with adjustable bases, to adapt to common parameter distributions. We show that our method exhibits superior performance at 4-bit precision on several performance benchmarks compared to asymmetric linear quantization at tensor-wise granularity, while achieving moderate speedup and high memory savings, making it suitable for private use on consumer-grade GPUs.

Problem

• quantization
• language models
• logarithmic space
• weight distribution
• memory efficiency

Innovation

• logarithmic quantization
• adjustable base
• 4-bit precision
• language model compression
• tensor-wise quantization