Semantics Meet Signals: Dual Codebook Representationl Learning for Generative Recommendation

2025-11-15

AI Summary

In generative recommendation, a unified codebook struggles to jointly model popular items (relying on collaborative signals) and long-tail items (requiring semantic understanding), leading to representation imbalance and limited generalization. To address this, we propose FlexCode—a novel collaborative-semantic dual-codebook framework that separately encodes these two signal types. FlexCode introduces a lightweight Mixture-of-Experts (MoE) routing mechanism to dynamically allocate token-level resources, and incorporates cross-popularity alignment and smoothing losses to enhance representation consistency. The method integrates discrete semantic tokenization, generative sequence modeling, contrastive learning, and multi-task optimization. Extensive experiments on both public and industrial-scale datasets demonstrate that FlexCode significantly outperforms strong baselines, achieving substantial gains in overall accuracy and long-tail item coverage.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Generative recommendation has recently emerged as a powerful paradigm that unifies retrieval and generation, representing items as discrete semantic tokens and enabling flexible sequence modeling with autoregressive models. Despite its success, existing approaches rely on a single, uniform codebook to encode all items, overlooking the inherent imbalance between popular items rich in collaborative signals and long-tail items that depend on semantic understanding. We argue that this uniform treatment limits representational efficiency and hinders generalization. To address this, we introduce FlexCode, a popularity-aware framework that adaptively allocates a fixed token budget between a collaborative filtering (CF) codebook and a semantic codebook. A lightweight MoE dynamically balances CF-specific precision and semantic generalization, while an alignment and smoothness objective maintains coherence across the popularity spectrum. We perform experiments on both public and industrial-scale datasets, showing that FlexCode consistently outperform strong baselines. FlexCode provides a new mechanism for token representation in generative recommenders, achieving stronger accuracy and tail robustness, and offering a new perspective on balancing memorization and generalization in token-based recommendation models.

Problem

• Addresses imbalance between popular and long-tail items in generative recommendation
• Introduces dual codebook framework to balance collaborative signals and semantic understanding
• Improves token representation for better accuracy and tail robustness in recommendation models

Innovation

• Dual codebook for collaborative and semantic signals
• Lightweight MoE adaptively balances token budget
• Alignment objective ensures coherence across popularity spectrum