Block-Sphere Vector Quantization

2026-05-19

AI Summary

This work addresses the lack of a unified theoretical framework in existing vector quantization methods, which often struggle to balance geometric fidelity and compression efficiency. We propose Block-Sphere Quantization (BlockQuant), a novel rotation-based block quantization paradigm grounded in spherical geometry. By applying spherical quantization to blocks of randomly rotated embedding vectors, BlockQuant more faithfully preserves the original geometric structure. Our theoretical analysis provides the first unified comparison of mainstream rotation-based quantizers, establishing BlockQuant’s superiority in terms of expected distortion and mean squared error. Empirical results demonstrate that BlockQuant consistently achieves significantly lower reconstruction error and higher inner product fidelity than state-of-the-art methods across real-world embedding datasets and long-context large language model inference tasks.

Abstract

Vector quantization is a fundamental primitive for scalable machine learning systems, enabling memory-efficient storage, fast retrieval, and compressed inference. Recent rotation-based quantizers such as EDEN, RabitQ, and TurboQuant have introduced strong guarantees and empirical performance, but the surrounding comparisons have been difficult to interpret because they rely on different distortion criteria, probability regimes, and implementation assumptions. As our first contribution, we provide a unified theoretical comparison of these methods and show that their relative advantages are criterion-dependent rather than absolute: EDEN and TurboQuant are favorable for MSE distortion, EDEN is also effective for expected inner-product distortion, and RabitQ provides strong high-probability control. This comparison further clarifies that EDEN provides particularly strong guarantees for expected distortion measures. As our second contribution, we introduce Block-Sphere Quantization (BlockQuant), a new rotation-based block quantization algorithm designed around the spherical geometry of randomly rotated vectors. Unlike coordinate-wise quantizers, BlockQuant quantizes blocks on the sphere, preserving the geometry of rotated embeddings more faithfully. We prove that this block-spherical design theoretically improves over the baselines considered in this paper for both reconstruction MSE and expected inner-product distortion. Our experiments on real embedding datasets and long-context LLM inference tasks show practical gains that are consistent with our theoretical improvements.

Problem

• vector quantization
• rotation-based quantization
• distortion criteria
• spherical geometry
• embedding compression

Innovation

• Block-Sphere Quantization
• rotation-based quantization
• vector quantization
• spherical geometry
• distortion guarantees