CARD: Non-Uniform Quantization of Visual Semantic Unit for Generative Recommendation

2026-04-29

AI Summary

This work addresses the insufficient supervision in semantic ID construction and the codebook imbalance and generation bias induced by uniform quantization in generative recommendation. To this end, the authors propose an end-to-end multimodal fusion framework that unifies textual, visual, and collaborative signals into structured visual-semantic units. A plug-and-play non-uniform transformation module is introduced, integrated with NU-RQ-VAE to enable learnable non-uniform vector quantization. This approach overcomes the limitations of conventional uniform quantization, substantially improving semantic modeling quality and codebook utilization. Extensive experiments demonstrate that the model outperforms state-of-the-art methods across multiple benchmark datasets, and the non-uniform transformation module exhibits strong generalization, readily adapting to diverse quantization schemes.

Abstract

Generative recommendation frameworks typically represent items as discrete Semantic IDs (SIDs). While existing studies have sought to enhance SID construction by incorporating multimodal content, collaborative signals, or more advanced quantization techniques, learning high-quality SIDs still faces two key challenges: (1) The two-stage generative recommendation paradigm (SID construction and autoregressive generation) provides insufficient supervision for heterogeneous fusion, which hinders learning high-quality SIDs, and (2) non-uniform embeddings lead to codeword imbalance and generation bias. To address these challenges, we propose a novel generative recommendation framework, called CARD. CARD introduces a visual semantic unit that unifies textual, visual, and collaborative signals into a structured visual representation prior to encoding, enabling holistic semantic modeling and effectively alleviating the semantic gap, thereby reducing the reliance on supervision signals during SID learning. Furthermore, to deal with the highly non-uniform distribution of item semantic embeddings in recommendation scenarios, we develop a non-uniform quantization framework (NU-RQ-VAE), which incorporates a learnable and invertible non-uniform transformation into the quantization process to map skewed semantic distributions into a more balanced latent space, thereby significantly improving codebook utilization and quantization accuracy. Experiments on multiple datasets show that CARD consistently outperforms baseline methods under various settings; meanwhile, the proposed non-uniform transformation module is plug-and-play and remains robust across different quantization schemes. Code is available at https://github.com/HAI-UESTC/CARD.

Problem

• generative recommendation
• semantic ID
• non-uniform quantization
• codebook imbalance
• heterogeneous fusion

Innovation

• non-uniform quantization
• visual semantic unit
• generative recommendation
• semantic ID
• NU-RQ-VAE