PolyGraph Discrepancy: a classifier-based metric for graph generation

2025-10-07

AI Summary

Existing graph generation models are commonly evaluated using Maximum Mean Discrepancy (MMD) computed over hand-crafted graph descriptors—yet MMD suffers from lacking an absolute scale, high sensitivity to kernel and descriptor parameters, and incomparability across different descriptors. To address these limitations, we propose PolyGraph Discrepancy (PGD), the first classifier-based, normalized metric for quantifying divergence between graph distributions. PGD leverages a trained graph-structure binary classifier within a variational lower-bound framework to estimate the Jensen–Shannon distance. Theoretically grounded, PGD yields a tighter lower bound on distributional divergence and is rigorously bounded in [0,1]. It exhibits robustness to kernel choice and descriptor parameterization, enabling principled cross-descriptor comparison. Extensive experiments demonstrate that PGD achieves superior discriminative power and stability compared to MMD, more accurately reflecting the true generative performance of graph models. Our implementation is publicly available.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Existing methods for evaluating graph generative models primarily rely on Maximum Mean Discrepancy (MMD) metrics based on graph descriptors. While these metrics can rank generative models, they do not provide an absolute measure of performance. Their values are also highly sensitive to extrinsic parameters, namely kernel and descriptor parametrization, making them incomparable across different graph descriptors. We introduce PolyGraph Discrepancy (PGD), a new evaluation framework that addresses these limitations. It approximates the Jensen-Shannon distance of graph distributions by fitting binary classifiers to distinguish between real and generated graphs, featurized by these descriptors. The data log-likelihood of these classifiers approximates a variational lower bound on the JS distance between the two distributions. Resulting metrics are constrained to the unit interval [0,1] and are comparable across different graph descriptors. We further derive a theoretically grounded summary metric that combines these individual metrics to provide a maximally tight lower bound on the distance for the given descriptors. Thorough experiments demonstrate that PGD provides a more robust and insightful evaluation compared to MMD metrics. The PolyGraph framework for benchmarking graph generative models is made publicly available at https://github.com/BorgwardtLab/polygraph-benchmark.

Problem

• Evaluating graph generative models lacks absolute performance measures
• Existing metrics are sensitive to kernel and descriptor parameters
• PolyGraph provides comparable metrics across different graph descriptors

Innovation

• Classifier-based metric for graph generation evaluation
• Approximates Jensen-Shannon distance via binary classifiers
• Combines descriptor metrics for tight theoretical bound