🤖 AI Summary
To address the limitations of fixed-bit quantization, accuracy degradation, and high query latency in Approximate k-Nearest Neighbor (AKNN) search within high-dimensional Euclidean spaces, this paper proposes Multi-Granularity Residual Quantization (MRQ). MRQ decouples the number of quantization bits from vector dimensionality for the first time, enhances distance correction accuracy via data distribution modeling, and integrates adaptive vector quantization, data-driven distance correction, efficient quantized distance computation, and error-bound optimization. Compared with state-of-the-art graph-based and quantization-based methods (e.g., RaBitQ), MRQ achieves a threefold speedup in query latency while maintaining identical retrieval accuracy and reducing code length to one-third. This significantly improves index configurability and practical applicability for large-scale AKNN search.
📝 Abstract
Approximate K nearest neighbor (AKNN) search in high-dimensional Euclidean space is a fundamental problem with widespread applications. Vector quantization which maps vectors to discrete quantized code, can significantly reduce the space cost of AKNN search while also accelerating the AKNN search speed. The exclusive use of vector quantization without precise vectors leads to a substantial decline in search accuracy. Recent research RaBitQ addresses this issue by using geometry relation to enhance quantization accuracy and employing error bound for distance correction with precise vector. However, this method requires that the quantization bit must be equal to the vector dimension resulting in a fixed compression ratio which limits its efficiency and flexibility. In this paper, we propose a new and efficient method MRQ to address this drawback. MRQ leverage leverages data distribution to achieve better distance correction and a higher vector compression ratio. MRQ reduces query latency using a highly efficient distance computation and correction scheme. Our results demonstrate that MRQ significantly outperforms state-of-the-art AKNN search methods based on graph or vector quantization, achieving up to a 3x efficiency speed-up with only 1/3 length of quantized code while maintaining the same accuracy.