Quality Measures for Dynamic Graph Generative Models

2025-03-03

AI Summary

Existing evaluation methods for dynamic graph generation rely on static snapshot discretization, failing to capture non-uniform temporal evolution, lacking a unified metric jointly sensitive to topology and node features, and suffering from incommensurable multi-dimensional metrics and high computational overhead due to explicit snapshot construction. This paper proposes the first end-to-end scalar similarity evaluation paradigm tailored for continuous-time dynamic graphs. Grounded in the Johnson–Lindenstrauss lemma, our framework employs randomized projection to enable snapshot-free, non-i.i.d. modeling and joint encoding of structural topology and node attributes. Extensive experiments on multiple real-world datasets demonstrate significant superiority over snapshot-based baselines, with over 10× speedup in evaluation latency and support for real-time assessment on graphs with up to ten million edges. The implementation is publicly available.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements

Abstract

Deep generative models have recently achieved significant success in modeling graph data, including dynamic graphs, where topology and features evolve over time. However, unlike in vision and natural language domains, evaluating generative models for dynamic graphs is challenging due to the difficulty of visualizing their output, making quantitative metrics essential. In this work, we develop a new quality metric for evaluating generative models of dynamic graphs. Current metrics for dynamic graphs typically involve discretizing the continuous-evolution of graphs into static snapshots and then applying conventional graph similarity measures. This approach has several limitations: (a) it models temporally related events as i.i.d. samples, failing to capture the non-uniform evolution of dynamic graphs; (b) it lacks a unified measure that is sensitive to both features and topology; (c) it fails to provide a scalar metric, requiring multiple metrics without clear superiority; and (d) it requires explicitly instantiating each static snapshot, leading to impractical runtime demands that hinder evaluation at scale. We propose a novel metric based on the extit{Johnson-Lindenstrauss} lemma, applying random projections directly to dynamic graph data. This results in an expressive, scalar, and application-agnostic measure of dynamic graph similarity that overcomes the limitations of traditional methods. We also provide a comprehensive empirical evaluation of metrics for continuous-time dynamic graphs, demonstrating the effectiveness of our approach compared to existing methods. Our implementation is available at https://github.com/ryienh/jl-metric.

Problem

• Develops a new metric for evaluating dynamic graph generative models.
• Addresses limitations of current metrics in capturing dynamic graph evolution.
• Proposes a scalar, application-agnostic measure using random projections.

Innovation

• Novel metric based on Johnson-Lindenstrauss lemma
• Random projections for dynamic graph similarity
• Scalar, expressive, application-agnostic measure