Expressivity of Representation Learning on Continuous-Time Dynamic Graphs: An Information-Flow Centric Review

2024-12-05

arXiv.org

AI Summary

This paper addresses the challenge of quantifying model expressiveness in continuous-time dynamic graph (CTDG) representation learning. We propose the first theoretical analysis framework grounded in the information flow (IF) perspective to characterize how models propagate and encode temporal and structural information. Methodologically, we systematically formulate a CTDG self-supervised learning paradigm—encompassing both predictive and contrastive approaches—and establish a taxonomy of methods aligned with graph types and downstream tasks, empirically validated on synthetic and real-world datasets. Key contributions include: (1) pioneering the integration of information flow modeling into theoretical expressiveness analysis for CTDGs; (2) revealing performance boundaries of existing methods on long-range dependencies, bipartite graphs, and community-structured graphs; and (3) providing interpretable, theory-backed guidelines for model selection and architectural design in CTDG learning.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Graphs are ubiquitous in real-world applications, ranging from social networks to biological systems, and have inspired the development of Graph Neural Networks (GNNs) for learning expressive representations. While most research has centered on static graphs, many real-world scenarios involve dynamic, temporally evolving graphs, motivating the need for Continuous-Time Dynamic Graph (CTDG) models. This paper provides a comprehensive review of Graph Representation Learning (GRL) on CTDGs with a focus on Self-Supervised Representation Learning (SSRL). We introduce a novel theoretical framework that analyzes the expressivity of CTDG models through an Information-Flow (IF) lens, quantifying their ability to propagate and encode temporal and structural information. Leveraging this framework, we categorize existing CTDG methods based on their suitability for different graph types and application scenarios. Within the same scope, we examine the design of SSRL methods tailored to CTDGs, such as predictive and contrastive approaches, highlighting their potential to mitigate the reliance on labeled data. Empirical evaluations on synthetic and real-world datasets validate our theoretical insights, demonstrating the strengths and limitations of various methods across long-range, bi-partite and community-based graphs. This work offers both a theoretical foundation and practical guidance for selecting and developing CTDG models, advancing the understanding of GRL in dynamic settings.

Problem

• Analyzing expressivity of CTDG models via Information-Flow framework
• Reviewing self-supervised representation learning methods for dynamic graphs
• Evaluating CTDG models on diverse graph types and applications

Innovation

• Novel Information-Flow framework for CTDG expressivity
• Self-Supervised Learning tailored for dynamic graphs
• Categorization of CTDG methods by graph types