DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs

📅 2024-08-13
🏛️ arXiv.org
📈 Citations: 11
Influential: 0
📄 PDF

career value

221K/year
🤖 AI Summary
To address weak temporal sensitivity and limited generalization in irregularly sampled dynamic graph modeling, this paper proposes a novel state-space model (SSM) controlled by continuous-time intervals. Inspired by the Ebbinghaus forgetting curve, we introduce a time-interval-driven mechanism into continuous-state modeling to robustly capture asynchronous temporal dynamics. Our method unifies time-interval-aware control signal design, dynamic graph embedding, and downstream tasks—including link prediction and node classification—within a single framework. Evaluated on 12 standard dynamic graph benchmarks, the approach achieves state-of-the-art (SOTA) performance on the majority of tasks while significantly reducing computational and memory overhead.

Technology Category

Application Category

📝 Abstract
Dynamic graph learning aims to uncover evolutionary laws in real-world systems, enabling accurate social recommendation (link prediction) or early detection of cancer cells (classification). Inspired by the success of state space models, e.g., Mamba, for efficiently capturing long-term dependencies in language modeling, we propose DyG-Mamba, a new continuous state space model (SSM) for dynamic graph learning. Specifically, we first found that using inputs as control signals for SSM is not suitable for continuous-time dynamic network data with irregular sampling intervals, resulting in models being insensitive to time information and lacking generalization properties. Drawing inspiration from the Ebbinghaus forgetting curve, which suggests that memory of past events is strongly correlated with time intervals rather than specific details of the events themselves, we directly utilize irregular time spans as control signals for SSM to achieve significant robustness and generalization. Through exhaustive experiments on 12 datasets for dynamic link prediction and dynamic node classification tasks, we found that DyG-Mamba achieves state-of-the-art performance on most of the datasets, while also demonstrating significantly improved computation and memory efficiency.
Problem

Research questions and friction points this paper is trying to address.

Modeling dynamic graphs to uncover evolutionary patterns
Improving model effectiveness and inductive capability using irregular timespans
Enhancing robustness by selectively reviewing historical information
Innovation

Methods, ideas, or system contributions that make the work stand out.

Translates dynamic graphs into long-term sequence modeling
Adjusts forgetting of historical information using irregular timespans
Selectively reviews history and filters noise for robustness