From Token to Item: Enhancing Large Language Models for Recommendation via Item-aware Attention Mechanism

2026-03-20

AI Summary

This work addresses a critical limitation in existing large language model (LLM)-based recommendation approaches, which predominantly focus on token-level interactions while overlooking the item as the fundamental unit of recommendation and thus fail to effectively capture collaborative relationships among items. To bridge this gap, the paper proposes an Item-Aware Attention Mechanism (IAM) tailored for recommendation tasks, which explicitly distinguishes and separately models intra-item and inter-item token relationships. Through a two-layer attention architecture, IAM captures semantic content within individual items and models collaborative signals across items, thereby elevating the item to the core unit of recommendation modeling. Extensive experiments on multiple public recommendation datasets demonstrate that IAM significantly outperforms current LLM-based methods, confirming its effectiveness and novelty.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Recommender Systems

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Models (LLMs) have recently gained increasing attention in the field of recommendation. Existing LLM-based methods typically represent items as token sequences, and apply attention layers on these tokens to generate recommendations. However, by inheriting the standard attention mechanism, these methods focus on modeling token-level relations. This token-centric focus overlooks the item as the fundamental unit of recommendation, preventing existing methods from effectively capturing collaborative relations at the item level. In this work, we revisit the role of tokens in LLM-driven recommendation and categorize their relations into two types: (1) intra-item token relations, which present the content semantics of an item, e.g., name, color, and size; and (2) inter-item token relations, which encode collaborative relations across items. Building on these insights, we propose a novel framework with an item-aware attention mechanism (IAM) to enhance LLMs for recommendation. Specifically, IAM devises two complementary attention layers: (1) an intra-item attention layer, which restricts attention to tokens within the same item, modeling item content semantics; and (2) an inter-item attention layer, which attends exclusively to token relations across items, capturing item collaborative relations. Through this stacked design, IAM explicitly emphasizes items as the fundamental units in recommendation, enabling LLMs to effectively exploit item-level collaborative relations. Extensive experiments on several public datasets demonstrate the effectiveness of IAM in enhancing LLMs for personalized recommendation.

Problem

• Large Language Models
• Recommendation
• Item-aware Attention
• Token-level Relations
• Collaborative Relations

Innovation

• item-aware attention
• large language models
• recommendation systems
• collaborative relations
• token-to-item modeling