Retrieval Augmented Generation for Dynamic Graph Modeling

2024-08-26

arXiv.org

AI Summary

Traditional dynamic graph modeling methods rely solely on target node histories, limiting their adaptability to emerging structures and evolutionary patterns. To address this, we introduce retrieval-augmented generation (RAG) into dynamic graph learning—the first such application—proposing a temporal-context-aware contrastive learning module and a multi-granularity graph fusion strategy. Our method retrieves semantically and temporally relevant high-quality historical examples globally to jointly enhance dynamic graph neural networks and inductive representation learning, thereby unifying support for both transductive and inductive generalization. Evaluated on multiple real-world dynamic graph datasets, our approach achieves significant improvements in link prediction and event prediction accuracy, while demonstrating strong robustness to unseen nodes and structural evolution. The code and datasets are publicly available.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Modeling dynamic graphs, such as those found in social networks, recommendation systems, and e-commerce platforms, is crucial for capturing evolving relationships and delivering relevant insights over time. Traditional approaches primarily rely on graph neural networks with temporal components or sequence generation models, which often focus narrowly on the historical context of target nodes. This limitation restricts the ability to adapt to new and emerging patterns in dynamic graphs. To address this challenge, we propose a novel framework, Retrieval-Augmented Generation for Dynamic Graph modeling (RAG4DyG), which enhances dynamic graph predictions by incorporating contextually and temporally relevant examples from broader graph structures. Our approach includes a time- and context-aware contrastive learning module to identify high-quality demonstrations and a graph fusion strategy to effectively integrate these examples with historical contexts. The proposed framework is designed to be effective in both transductive and inductive scenarios, ensuring adaptability to previously unseen nodes and evolving graph structures. Extensive experiments across multiple real-world datasets demonstrate the effectiveness of RAG4DyG in improving predictive accuracy and adaptability for dynamic graph modeling. The code and datasets are publicly available at https://github.com/YuxiaWu/RAG4DyG.

Problem

• Modeling evolving relationships in dynamic graphs effectively
• Overcoming limitations of traditional historical context-focused methods
• Enhancing adaptability to new patterns in dynamic graphs

Innovation

• Retrieval-Augmented Generation for dynamic graphs
• Time-aware contrastive learning module
• Graph fusion strategy for integration