Quantization Beyond Uniform Bit Allocation

2026-08-19

AI Summary

研究通过非均匀位分配方案改进嵌入量化质量，提出一种可变位分配框架，并在低比特率下显著提高召回率。

Abstract

Quantization is a fundamental technique to handle the growing sizes of embeddings generated by modern models. Existing quantization schemes are largely embedding agnostic and allocate bits uniformly across dimensions. However, recent models produce embeddings with significant geometric structure. In this work, we investigate whether a variable bit allocation scheme can improve quantization quality under a fixed memory budget. We propose a simple variable bit allocation framework that partitions an embedding into contiguous buckets and allocates storage non-uniformly across them. Using a greedy allocation strategy, we instantiate this framework for both Product Quantization (PQ) and Scalar Quantization (SQ). We perform a series of experiments on embeddings known to have the Matryoshka property (MRL), and consistently observe that non-uniform allocations outperform uniform baselines at identical storage budgets. The largest improvements occur in the low-bit regime, where uniform allocation is particularly inefficient for MRL embeddings. At the same compression rates, variable allocation improves recall by up to 8\% for PQ and up to 18\% for SQ. Our results suggest a new direction for structure-aware compression and indexing techniques for large-scale retrieval systems.

Problem

• Quantization
• Variable Bit Allocation
• Embedding Geometry
• Memory Budget
• Compression Quality

Innovation

• Variable Bit Allocation
• Non-Uniform Storage
• Embedding Quantization
• Matryoshka Property