A Unified Rate-Distortion Perspective on Vector, Product, and Scalar Quantization

2026-09-02

AI Summary

本文提出了一种统一的率-失真视角来理解向量、乘积和标量量化，解决了视觉标记化中的量化权衡问题。

Abstract

Discrete visual tokenization, predominantly driven by vector, scalar, and product quantization, lacks a unified conceptual framework for understanding quantization tradeoffs. In this paper, we propose a unified rate--distortion perspective on modern discrete visual tokenization. By viewing quantization as lossy compression, we characterize the nominal fixed-length coding rate through token count and codebook size, and quantization error as the distortion. Within this framework, we resolve three central questions. First, we theoretically and empirically show that minimizing distortion, rather than maximizing codebook utilization, is the primary intrinsic objective for reconstruction fidelity, with a direct connection to the STE-induced gradient discrepancy. Second, we establish two critical fairness conditions for intrinsic quantization comparison: controlling latent feature statistics and enforcing identical coding rates. Third, under these conditions, we recover the VQ--PQ--SQ distortion hierarchy in modern visual tokenization and show empirically that modern VQ methods achieve the lowest distortion. This work provides a foundational rate--distortion reframing of modern discrete visual tokenization, resolves ambiguities in quantizer evaluation, and provides a controlled framework for isolating intrinsic quantization effectiveness under fixed-rate constraints.

Problem

• vector quantization
• product quantization
• scalar quantization
• rate-distortion
• visual tokenization

Innovation

• Rate-Distortion Perspective
• Quantization Tradeoffs
• Visual Tokenization
• Fixed-Length Coding Rate
• Distortion Hierarchy