Making Collaborative Signals Count: Graph-Aware Large Language Models for Sequential Recommendation

2026-08-12

AI Summary

This work addresses the limitation of large language models (LLMs) in sequential recommendation, where language-centric pretraining fails to capture collaborative signals from user–item interactions, thereby hindering personalized performance. To overcome this, the authors propose a lightweight approach that constructs a collaborative graph integrating textual and item tokens, explicitly modeling text–text, item–text, and item–item relationships. These relational structures are converted into learnable attention biases and directly injected into the LLM’s attention mechanism, enabling global collaborative awareness without requiring an external graph encoder. This is the first method to incorporate global collaborative signals into LLMs via attention biases, circumventing the reliance of existing approaches on local interaction sequences or auxiliary recommender systems. Experiments on four real-world datasets demonstrate an average 9.76% improvement in HR@5 over strong baselines, highlighting its effectiveness.

Abstract

Large language models (LLMs) have been widely adopted as backbones for recommender systems. However, their language-centric pretraining makes it difficult to capture collaborative signals implicit in user-item interactions, which are crucial for personalized recommendation. Existing methods either inject collaborative representations produced by external recommenders or model only intra-sequence dependencies, limiting their ability to exploit global collaborative patterns. To address this limitation, we propose GALLM, a graph-aware LLM framework for sequential recommendation. GALLM constructs a collaborative graph over text tokens and item tokens, and models three types of relations: Text--Text relations for preserving semantic dependencies, Item--Text relations for aligning item tokens with their textual descriptions, and Item--Item relations derived from global item co-occurrence patterns. These relations are transformed into lightweight learnable attention biases and incorporated into the LLM attention mechanism, enabling collaborative-aware token interactions without introducing an additional graph encoder. Experiments on four real-world benchmarks show that GALLM achieves the best performance among the compared baselines, improving over the strongest baseline by 9.76\% on average in HR@5.

Problem

• large language models
• sequential recommendation
• collaborative signals
• user-item interactions
• personalized recommendation

Innovation

• graph-aware LLM
• collaborative signals
• sequential recommendation
• attention bias
• item co-occurrence