Virtual Nodes Can Help: Tackling Distribution Shifts in Federated Graph Learning

2024-12-26

AI Summary

To address performance degradation of graph neural networks (GNNs) in federated graph learning (FGL) caused by heterogeneous graph data distributions across clients, this paper proposes FedVN—a novel framework featuring learnable virtual nodes (VNs) and client-specific edge generators. FedVN establishes a virtual-node-driven adaptive graph augmentation mechanism that jointly aligns local graph distributions and facilitates global model co-training. Theoretically, we prove that this mechanism effectively mitigates distribution shift while enabling joint optimization of shared VNs and personalized edge connection strategies. Extensive experiments on four benchmark datasets under five distinct distribution shift settings demonstrate that FedVN consistently outperforms nine state-of-the-art baselines, achieving an average 12.7% improvement in graph property prediction tasks. To the best of our knowledge, FedVN is the first framework to realize distribution-aware federated graph learning grounded in virtual nodes.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Federated Graph Learning (FGL) enables multiple clients to jointly train powerful graph learning models, e.g., Graph Neural Networks (GNNs), without sharing their local graph data for graph-related downstream tasks, such as graph property prediction. In the real world, however, the graph data can suffer from significant distribution shifts across clients as the clients may collect their graph data for different purposes. In particular, graph properties are usually associated with invariant label-relevant substructures (i.e., subgraphs) across clients, while label-irrelevant substructures can appear in a client-specific manner. The issue of distribution shifts of graph data hinders the efficiency of GNN training and leads to serious performance degradation in FGL. To tackle the aforementioned issue, we propose a novel FGL framework entitled FedVN that eliminates distribution shifts through client-specific graph augmentation strategies with multiple learnable Virtual Nodes (VNs). Specifically, FedVN lets the clients jointly learn a set of shared VNs while training a global GNN model. To eliminate distribution shifts, each client trains a personalized edge generator that determines how the VNs connect local graphs in a client-specific manner. Furthermore, we provide theoretical analyses indicating that FedVN can eliminate distribution shifts of graph data across clients. Comprehensive experiments on four datasets under five settings demonstrate the superiority of our proposed FedVN over nine baselines.

Problem

• Federated Graph Learning
• Distribution Shift
• Graph Neural Network

Innovation

• FedVN
• Federated Graph Learning
• Virtual Node Adjustment