A Comprehensive Graph Pooling Benchmark: Effectiveness, Robustness and Generalizability

2024-06-13

arXiv.org

AI Summary

Existing graph pooling methods lack standardized, multi-dimensional fairness benchmarks for systematic evaluation. Method: We introduce the first unified evaluation framework covering 17 pooling methods across 28 graph datasets, assessing effectiveness (classification, regression, node-level tasks), robustness (noise injection, adversarial perturbations), and generalization (distribution shift, out-of-distribution generalization), while incorporating efficiency, backbone compatibility (GCN, GAT, GIN), and parameter sensitivity analysis. We conduct extensive experiments with canonical poolers (e.g., TopK, SAGPool, ASAP, DiffPool) and provide visualization and ablation studies. Contribution/Results: Our framework reveals performance boundaries across task types, graph scales, and perturbation regimes. The open-sourced code and benchmark fill a critical gap in graph learning, enabling reproducible, comparable, and fair research—establishing a new standard for evaluating graph pooling methods.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Graph pooling has gained attention for its ability to obtain effective node and graph representations for various downstream tasks. Despite the recent surge in graph pooling approaches, there is a lack of standardized experimental settings and fair benchmarks to evaluate their performance. To address this issue, we have constructed a comprehensive benchmark that includes 17 graph pooling methods and 28 different graph datasets. This benchmark systematically assesses the performance of graph pooling methods in three dimensions, i.e., effectiveness, robustness, and generalizability. We first evaluate the performance of these graph pooling approaches across different tasks including graph classification, graph regression and node classification. Then, we investigate their performance under potential noise attacks and out-of-distribution shifts in real-world scenarios. We also involve detailed efficiency analysis, backbone analysis, parameter analysis and visualization to provide more evidence. Extensive experiments validate the strong capability and applicability of graph pooling approaches in various scenarios, which can provide valuable insights and guidance for deep geometric learning research. The source code of our benchmark is available at https://github.com/goose315/Graph_Pooling_Benchmark.

Problem

• Lack of standardized benchmarks for graph pooling evaluation
• Systematic assessment of pooling methods across multiple dimensions
• Comprehensive evaluation of 17 methods on 28 graph datasets

Innovation

• Constructed benchmark with 17 graph pooling methods
• Systematically assessed performance across three dimensions
• Evaluated methods under noise attacks and distribution shifts