HMamba: Hyperbolic Mamba for Sequential Recommendation

2025-05-14

AI Summary

Existing sequential recommendation models (e.g., RNNs/Transformers) suffer from high computational overhead and struggle to capture the inherent hierarchical structure of user preferences; while Mamba-based models achieve linear time complexity, they operate in Euclidean space and thus fail to model the hyperbolic geometry naturally exhibited by recommendation data. To address these limitations, we propose HyperMamba—the first end-to-end sequential recommendation framework integrating hyperbolic geometry with selective state space modeling (SSM). Its core innovations include: (i) a selective state update mechanism formulated in hyperbolic space, (ii) a stable Riemannian gradient optimization algorithm, and (iii) a Poincaré ball embedding and projection strategy. Evaluated on four benchmark datasets, HyperMamba achieves 3–11% improvements in Recall@20 while retaining O(L) time complexity, enabling scalable, real-time industrial deployment.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Learning with Manifolds
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Sequential recommendation systems have become a cornerstone of personalized services, adept at modeling the temporal evolution of user preferences by capturing dynamic interaction sequences. Existing approaches predominantly rely on traditional models, including RNNs and Transformers. Despite their success in local pattern recognition, Transformer-based methods suffer from quadratic computational complexity and a tendency toward superficial attention patterns, limiting their ability to infer enduring preference hierarchies in sequential recommendation data. Recent advances in Mamba-based sequential models introduce linear-time efficiency but remain constrained by Euclidean geometry, failing to leverage the intrinsic hyperbolic structure of recommendation data. To bridge this gap, we propose Hyperbolic Mamba, a novel architecture that unifies the efficiency of Mamba's selective state space mechanism with hyperbolic geometry's hierarchical representational power. Our framework introduces (1) a hyperbolic selective state space that maintains curvature-aware sequence modeling and (2) stabilized Riemannian operations to enable scalable training. Experiments across four benchmarks demonstrate that Hyperbolic Mamba achieves 3-11% improvement while retaining Mamba's linear-time efficiency, enabling real-world deployment. This work establishes a new paradigm for efficient, hierarchy-aware sequential modeling.

Problem

• Overcome quadratic complexity and superficial attention in Transformers
• Address Euclidean geometry limitations in Mamba-based models
• Model hierarchical preferences in sequential recommendation data

Innovation

• Hyperbolic Mamba combines Mamba's efficiency with hyperbolic geometry
• Introduces hyperbolic selective state space for curvature-aware modeling
• Uses stabilized Riemannian operations for scalable training