Unlocking the Power of Diffusion Models in Sequential Recommendation: A Simple and Effective Approach

2025-05-26

AI Summary

To address the pervasive embedding collapse problem in diffusion-based sequential recommendation, this paper proposes ADRec—a novel framework featuring a token-level independent noise injection mechanism that enables joint diffusion training over entire sequences. ADRec integrates autoregressive modeling with token-level distribution learning and introduces a three-stage progressive training strategy to mitigate embedding collapse. During inference, it performs denoising exclusively on the final token, thereby preserving historical interaction patterns while significantly improving computational efficiency. Evaluated on six public benchmark datasets, ADRec achieves substantial improvements in recommendation accuracy, yielding average gains of +3.2% in Recall@10 and NDCG@10, while reducing inference latency by 42%. To the best of our knowledge, ADRec is the first diffusion-based sequential recommender system to achieve simultaneous optimization of both accuracy and efficiency.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

In this paper, we focus on the often-overlooked issue of embedding collapse in existing diffusion-based sequential recommendation models and propose ADRec, an innovative framework designed to mitigate this problem. Diverging from previous diffusion-based methods, ADRec applies an independent noise process to each token and performs diffusion across the entire target sequence during training. ADRec captures token interdependency through auto-regression while modeling per-token distributions through token-level diffusion. This dual approach enables the model to effectively capture both sequence dynamics and item representations, overcoming the limitations of existing methods. To further mitigate embedding collapse, we propose a three-stage training strategy: (1) pre-training the embedding weights, (2) aligning these weights with the ADRec backbone, and (3) fine-tuning the model. During inference, ADRec applies the denoising process only to the last token, ensuring that the meaningful patterns in historical interactions are preserved. Our comprehensive empirical evaluation across six datasets underscores the effectiveness of ADRec in enhancing both the accuracy and efficiency of diffusion-based sequential recommendation systems.

Problem

• Addressing embedding collapse in diffusion-based sequential recommendation models
• Proposing ADRec to capture sequence dynamics and item representations
• Enhancing accuracy and efficiency in sequential recommendation systems

Innovation

• Independent noise process per token
• Three-stage training strategy
• Denoising only last token during inference