Catalog-Native LLM: Speaking Item-ID Dialect with Less Entanglement for Recommendation

2025-09-30

AI Summary

Recommender systems face a fundamental challenge: reconciling the accuracy of collaborative filtering with the semantic understanding and generalization capabilities of large language models (LLMs), while overcoming the semantic opacity of collaborative signals and the inability of text-only LLMs to model implicit user preferences. To address this, we propose IDIOMoE—a novel architecture that treats item-ID sequences as “native dialects” within the language space. It introduces a token-type-gated Mixture-of-Experts (MoE) feed-forward network, explicitly separating textual experts from item-ID experts to prevent cross-modal representation entanglement. This design preserves the pre-trained LLM’s linguistic competence while seamlessly integrating collaborative signals. Extensive experiments on multiple public and proprietary datasets demonstrate significant improvements in recommendation accuracy. Moreover, IDIOMoE natively supports natural-language querying and enables interpretable recommendations through its modular, semantically grounded architecture.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Data Mining & Knowledge Management: Recommender Systems
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

While collaborative filtering delivers predictive accuracy and efficiency, and Large Language Models (LLMs) enable expressive and generalizable reasoning, modern recommendation systems must bring these strengths together. Growing user expectations, such as natural-language queries and transparent explanations, further highlight the need for a unified approach. However, doing so is nontrivial. Collaborative signals are often token-efficient but semantically opaque, while LLMs are semantically rich but struggle to model implicit user preferences when trained only on textual inputs. This paper introduces Item-ID + Oral-language Mixture-of-Experts Language Model (IDIOMoE), which treats item interaction histories as a native dialect within the language space, enabling collaborative signals to be understood in the same way as natural language. By splitting the Feed Forward Network of each block of a pretrained LLM into a separate text expert and an item expert with token-type gating, our method avoids destructive interference between text and catalog modalities. IDIOMoE demonstrates strong recommendation performance across both public and proprietary datasets, while preserving the text understanding of the pretrained model.

Problem

• Integrating collaborative filtering accuracy with LLM reasoning for recommendations
• Resolving semantic opacity of collaborative signals in recommendation systems
• Preventing interference between text and catalog modalities in unified models

Innovation

• Item-ID dialect integrates collaborative signals into language space
• Mixture-of-Experts separates text and item experts with gating
• Preserves pretrained model's text understanding while enhancing recommendations