OpenFGL: A Comprehensive Benchmarks for Federated Graph Learning

2024-08-29

arXiv.org

AI Summary

To address the lack of standardized evaluation protocols in federated graph learning (FGL), this paper introduces OpenFGL—the first open-source benchmark for FGL. It systematically encompasses both Graph-FL and Subgraph-FL paradigms, integrating 38 cross-domain graph datasets, 8 graph-aware federated partitioning strategies, 5 downstream task categories, and APIs for 18 state-of-the-art algorithms. By establishing a privacy-preserving, distributed GNN training and evaluation framework, OpenFGL enables the first standardized, reproducible comparison of FGL methods across effectiveness, robustness, and efficiency. Empirical analysis uncovers common limitations of existing approaches under graph heterogeneity, sparsity, and label skew, highlighting critical challenges in real-world deployment. OpenFGL thus provides a scalable benchmark platform and a reproducible evaluation protocol to advance principled research and development in federated graph learning.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Federated graph learning (FGL) has emerged as a promising distributed training paradigm for graph neural networks across multiple local systems without direct data sharing. This approach is particularly beneficial in privacy-sensitive scenarios and offers a new perspective on addressing scalability challenges in large-scale graph learning. Despite the proliferation of FGL, the diverse motivations from practical applications, spanning various research backgrounds and experimental settings, pose a significant challenge to fair evaluation. To fill this gap, we propose OpenFGL, a unified benchmark designed for the primary FGL scenarios: Graph-FL and Subgraph-FL. Specifically, OpenFGL includes 38 graph datasets from 16 application domains, 8 federated data simulation strategies that emphasize graph properties, and 5 graph-based downstream tasks. Additionally, it offers 18 recently proposed SOTA FGL algorithms through a user-friendly API, enabling a thorough comparison and comprehensive evaluation of their effectiveness, robustness, and efficiency. Empirical results demonstrate the ability of FGL while also revealing its potential limitations, offering valuable insights for future exploration in this thriving field.

Problem

• Federated Graph Learning
• Unified Evaluation Standard
• Diverse Application Demands

Innovation

• OpenFGL
• Federal Graph Learning
• Comprehensive Evaluation Framework