Training Large Recommendation Models via Graph-Language Token Alignment

2025-02-26

AI Summary

Traditional recommender systems struggle to jointly model graph-structured interaction signals and textual semantics, while direct invocation of large language models (LLMs) suffers from generation ambiguity and low inference efficiency. To address these challenges, we propose the Graph–Language Token Alignment (GLTA) and Graph–Language Logical Matching (GLLM) framework, which maps user/item node embeddings into the LLM’s semantic space, enabling end-to-end differentiable alignment between graph structure and text semantics. Our approach integrates graph neural networks, LLM embedding alignment, contrastive learning, and logits-level supervised matching—bypassing autoregressive text generation entirely. This design significantly improves recommendation accuracy and generalization. Extensive experiments on three benchmark datasets demonstrate consistent superiority over state-of-the-art methods. Ablation studies validate the effectiveness and necessity of each component.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recommender systems (RS) have become essential tools for helping users efficiently navigate the overwhelming amount of information on e-commerce and social platforms. However, traditional RS relying on Collaborative Filtering (CF) struggles to integrate the rich semantic information from textual data. Meanwhile, large language models (LLMs) have shown promising results in natural language processing, but directly using LLMs for recommendation introduces challenges, such as ambiguity in generating item predictions and inefficiencies in scalability. In this paper, we propose a novel framework to train Large Recommendation models via Graph-Language Token Alignment. By aligning item and user nodes from the interaction graph with pretrained LLM tokens, GLTA effectively leverages the reasoning abilities of LLMs. Furthermore, we introduce Graph-Language Logits Matching (GLLM) to optimize token alignment for end-to-end item prediction, eliminating ambiguity in the free-form text as recommendation results. Extensive experiments on three benchmark datasets demonstrate the effectiveness of GLTA, with ablation studies validating each component.

Problem

• Integrate textual semantic in recommendation
• Overcome ambiguity in LLMs for item prediction
• Enhance scalability in recommendation systems

Innovation

• Graph-Language Token Alignment
• Graph-Language Logits Matching
• Pretrained LLM tokens integration