STAR-Rec: Making Peace with Length Variance and Pattern Diversity in Sequential Recommendation

2025-05-06

AI Summary

To address performance degradation in recommender systems caused by variable-length user behavior sequences and heterogeneous interaction patterns, this paper proposes a sequence-level Mixture-of-Experts (MoE) architecture that— for the first time in recommendation—synergistically integrates preference-aware attention with State Space Models (SSMs). SSMs efficiently capture long-range temporal dynamics, while attention simultaneously models user-item similarity and behavioral diversity. A behavior-aware routing mechanism is introduced to adaptively distinguish browsing from exploratory interaction modes. Grounded in theoretical analysis, the attention-SSM co-modeling framework ensures functional complementarity and interpretability. Extensive experiments on four real-world datasets demonstrate consistent superiority over state-of-the-art methods, particularly in challenging scenarios involving long-tail interactions, short sequences, and cross-category exploration—achieving an average +3.2% improvement in Recall@10.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Data Mining & Knowledge Management: Recommender Systems
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent deep sequential recommendation models often struggle to effectively model key characteristics of user behaviors, particularly in handling sequence length variations and capturing diverse interaction patterns. We propose STAR-Rec, a novel architecture that synergistically combines preference-aware attention and state-space modeling through a sequence-level mixture-of-experts framework. STAR-Rec addresses these challenges by: (1) employing preference-aware attention to capture both inherently similar item relationships and diverse preferences, (2) utilizing state-space modeling to efficiently process variable-length sequences with linear complexity, and (3) incorporating a mixture-of-experts component that adaptively routes different behavioral patterns to specialized experts, handling both focused category-specific browsing and diverse category exploration patterns. We theoretically demonstrate how the state space model and attention mechanisms can be naturally unified in recommendation scenarios, where SSM captures temporal dynamics through state compression while attention models both similar and diverse item relationships. Extensive experiments on four real-world datasets demonstrate that STAR-Rec consistently outperforms state-of-the-art sequential recommendation methods, particularly in scenarios involving diverse user behaviors and varying sequence lengths.

Problem

• Handling sequence length variations in sequential recommendation
• Capturing diverse user interaction patterns effectively
• Unifying state-space modeling and attention mechanisms adaptively

Innovation

• Preference-aware attention captures similar and diverse item relationships
• State-space modeling processes variable-length sequences efficiently
• Mixture-of-experts adaptively routes diverse behavioral patterns