SIGMA: Selective Gated Mamba for Sequential Recommendation

2024-08-21

AI Summary

Existing sequential recommendation models suffer from limitations in bidirectional interaction modeling, short-term pattern capture, and state estimation stability. To address these issues, this paper proposes PF-Mamba, a bidirectional sequential recommendation framework built upon the Mamba architecture. Its key contributions are: (1) a novel Partial Flip mechanism enabling efficient bidirectional contextual modeling; (2) an input-sensitive Dense Selection (DS) Gate that dynamically fuses forward and backward hidden states; and (3) a lightweight Feature Extraction GRU (FE-GRU) to strengthen short-sequence dependency modeling. Extensive experiments on five real-world datasets demonstrate that PF-Mamba consistently outperforms state-of-the-art methods, achieving significant improvements in Recall@10 and Mean Reciprocal Rank (MRR), while reducing inference latency. The source code is publicly available.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Mixture of Experts (MoE)
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

In various domains, Sequential Recommender Systems (SRS) have become essential due to their superior capability to discern intricate user preferences. Typically, SRS utilize transformer-based architectures to forecast the subsequent item within a sequence. Nevertheless, the quadratic computational complexity inherent in these models often leads to inefficiencies, hindering the achievement of real-time recommendations. Mamba, a recent advancement, has exhibited exceptional performance in time series prediction, significantly enhancing both efficiency and accuracy. However, integrating Mamba directly into SRS poses several challenges. Its inherently unidirectional nature may constrain the model's capacity to capture the full context of user-item interactions, while its instability in state estimation can compromise its ability to detect short-term patterns within interaction sequences. To overcome these issues, we introduce a new framework named Selective Gated Mamba (SIGMA) for Sequential Recommendation. This framework leverages a Partially Flipped Mamba (PF-Mamba) to construct a bidirectional architecture specifically tailored to improve contextual modeling. Additionally, an input-sensitive Dense Selective Gate (DS Gate) is employed to optimize directional weights and enhance the processing of sequential information in PF-Mamba. For short sequence modeling, we have also developed a Feature Extract GRU (FE-GRU) to efficiently capture short-term dependencies. Empirical results indicate that SIGMA outperforms current models on five real-world datasets. Our implementation code is available at https://github.com/ziwliu-cityu/SIMGA to ease reproducibility.

Problem

• Sequence Recommendation
• Bidirectional Interaction
• Short-term Pattern

Innovation

• SIGMA Framework
• Bi-directional PF-Mamba Model
• FE-GRU