Benchmarking Spectral Graph Neural Networks: A Comprehensive Study on Effectiveness and Efficiency

2024-06-14

arXiv.org

AI Summary

Existing spectral graph neural networks (Spectral GNNs) suffer from a lack of unified, fair, and systematic evaluation—particularly across efficiency, memory consumption, and predictive effectiveness—due to inconsistent implementations and missing standardized benchmarks. To address this, we propose the first comprehensive evaluation framework specifically designed for spectral-domain GNNs, featuring a standardized implementation infrastructure and an extensible assessment protocol. Our framework integrates over 30 Spectral GNN models and 27 frequency-domain filters, enabling multi-dimensional quantitative analysis under consistent training and inference paradigms. We further introduce customized graph computation kernels, sparse matrix optimizations, and memory-efficient training strategies. Extensive experiments on standard benchmarks and large-scale graphs demonstrate up to 3.2× faster inference and 41% reduced memory overhead. The codebase is publicly released.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

With the recent advancements in graph neural networks (GNNs), spectral GNNs have received increasing popularity by virtue of their specialty in capturing graph signals in the frequency domain, demonstrating promising capability in specific tasks. However, few systematic studies have been conducted on assessing their spectral characteristics. This emerging family of models also varies in terms of designs and settings, leading to difficulties in comparing their performance and deciding on the suitable model for specific scenarios, especially for large-scale tasks. In this work, we extensively benchmark spectral GNNs with a focus on the frequency perspective. We analyze and categorize over 30 GNNs with 27 corresponding filters. Then, we implement these spectral models under a unified framework with dedicated graph computations and efficient training schemes. Thorough experiments are conducted on the spectral models with inclusive metrics on effectiveness and efficiency, offering practical guidelines on evaluating and selecting spectral GNNs with desirable performance. Our implementation enables application on larger graphs with comparable performance and less overhead, which is available at: https://github.com/gdmnl/Spectral-GNN-Benchmark.

Problem

• Benchmarking spectral GNNs' effectiveness and efficiency systematically
• Evaluating spectral models for deployment on massive web-scale graphs
• Analyzing spectral graph filters across diverse formulations and utilizations

Innovation

• Benchmarking spectral GNNs with unified framework
• Implementing 27 spectral filters for diverse graphs
• Enabling deployment on million-scale graphs efficiently