Leveraging Historical and Current Interests for Continual Sequential Recommendation

2025-06-09

AI Summary

To address catastrophic forgetting, high computational overhead, and cold-start user learning challenges in sequential recommendation over non-stationary data streams, this paper proposes CSA-Rec, a continual learning-based sequential recommendation model. Methodologically, it introduces (1) Continual Sequential Attention (CSA), a linear attention mechanism that jointly preserves long-term historical interests and captures evolving short-term preferences; (2) Cauchy–Schwarz normalization to enhance training stability; (3) a collaborative learnable interest pool to mitigate forgetting; and (4) a user-similarity-driven cross-user knowledge transfer mechanism to improve cold-start robustness. Evaluated on three real-world streaming datasets, CSA-Rec significantly outperforms state-of-the-art methods, achieving superior performance in both knowledge retention rate and novel interest acquisition while reducing computational complexity.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Data Mining & Knowledge Management: Recommender Systems
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

Sequential recommendation models based on the Transformer architecture show superior performance in harnessing long-range dependencies within user behavior via self-attention. However, naively updating them on continuously arriving non-stationary data streams incurs prohibitive computation costs or leads to catastrophic forgetting. To address this, we propose Continual Sequential Transformer for Recommendation (CSTRec) that effectively leverages well-preserved historical user interests while capturing current interests. At its core is Continual Sequential Attention (CSA), a linear attention mechanism that retains past knowledge without direct access to old data. CSA integrates two key components: (1) Cauchy-Schwarz Normalization that stabilizes training under uneven interaction frequencies, and (2) Collaborative Interest Enrichment that mitigates forgetting through shared, learnable interest pools. We further introduce a technique that facilitates learning for cold-start users by transferring historical knowledge from behaviorally similar existing users. Extensive experiments on three real-world datasets indicate that CSTRec outperforms state-of-the-art baselines in both knowledge retention and acquisition.

Problem

• Addresses catastrophic forgetting in sequential recommendation models
• Proposes a method to leverage historical and current user interests
• Improves recommendation for cold-start users via knowledge transfer

Innovation

• Linear attention mechanism retains past knowledge
• Cauchy-Schwarz Normalization stabilizes training
• Collaborative Interest Enrichment mitigates forgetting