STORE: Streamlining Semantic Tokenization and Generative Recommendation with A Single LLM

2024-09-11

arXiv.org

AI Summary

Traditional recommender systems rely on item IDs, limiting semantic modeling and generalization to cold-start and long-tail scenarios. To address this, we propose STORE—the first framework unifying semantic tokenization (text-to-semantic-token) and generative recommendation (token-to-token) within a single large language model (LLM), eliminating fragile quantization modules such as RQ-VAE. STORE employs multi-stage generative modeling: jointly optimizing token-to-token prediction, token-to-text reconstruction, and text-to-token auxiliary tasks for end-to-end semantic understanding and recommendation. By removing modular coupling, STORE achieves superior semantic expressiveness, significantly improving coverage of long-tail items and cold-start performance. It outperforms state-of-the-art methods across multiple benchmarks. The code and configurations are publicly available.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Traditional recommendation models often rely on unique item identifiers (IDs) to distinguish between items, which can hinder their ability to effectively leverage item content information and generalize to long-tail or cold-start items. Recently, semantic tokenization has been proposed as a promising solution that aims to tokenize each item's semantic representation into a sequence of discrete tokens. In this way, it preserves the item's semantics within these tokens and ensures that semantically similar items are represented by similar tokens. These semantic tokens have become fundamental in training generative recommendation models. However, existing generative recommendation methods typically involve multiple sub-models for embedding, quantization, and recommendation, leading to an overly complex system. In this paper, we propose to streamline the semantic tokenization and generative recommendation process with a unified framework, dubbed STORE, which leverages a single large language model (LLM) for both tasks. Specifically, we formulate semantic tokenization as a text-to-token task and generative recommendation as a token-to-token task, supplemented by a token-to-text reconstruction task and a text-to-token auxiliary task. All these tasks are framed in a generative manner and trained using a single LLM backbone. Extensive experiments have been conducted to validate the effectiveness of our STORE framework across various recommendation tasks and datasets. We will release the source code and configurations for reproducible research.

Problem

• Overcoming limitations of unique item IDs in recommendation models
• Improving semantic tokenization for generative recommendation systems
• Addressing challenges in embedding extraction and training stability

Innovation

• Multi-aspect semantic tokenization via item palette
• Domain-specific tuning for semantic encoders
• Improved embedding extraction and training stability