Decoupled Subgraph Federated Learning

2024-02-29

International Conference on Learning Representations

AI Summary

In federated learning over distributed subgraphs, there exists a fundamental tension between modeling structural dependencies across clients and preserving privacy—specifically, how to effectively capture cross-client node dependencies via the global graph structure without sharing or generating sensitive node features or embeddings. Method: We propose FedStruct, the first decoupled subgraph federated learning framework, featuring (i) a global-topology-guided local collaborative update mechanism, (ii) a structure-aware gradient alignment strategy, and (iii) a feature-exchange-free dependency distillation method across subgraphs. Results: Extensive experiments on six semi-supervised node classification benchmarks demonstrate that FedStruct achieves performance comparable to centralized training, while exhibiting strong robustness to diverse subgraph partitioning schemes, label sparsity, and client scale. To our knowledge, FedStruct is the first approach to simultaneously achieve high privacy guarantees and high structural integrity in graph federated learning.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

We address the challenge of federated learning on graph-structured data distributed across multiple clients. Specifically, we focus on the prevalent scenario of interconnected subgraphs, where interconnections between different clients play a critical role. We present a novel framework for this scenario, named FedStruct, that harnesses deep structural dependencies. To uphold privacy, unlike existing methods, FedStruct eliminates the necessity of sharing or generating sensitive node features or embeddings among clients. Instead, it leverages explicit global graph structure information to capture inter-node dependencies. We validate the effectiveness of FedStruct through experimental results conducted on six datasets for semi-supervised node classification, showcasing performance close to the centralized approach across various scenarios, including different data partitioning methods, varying levels of label availability, and number of clients.

Problem

• Federated learning on distributed graph-structured data
• Handling interconnected subgraphs across multiple clients
• Privacy-preserving node classification without sharing sensitive data

Innovation

• Decoupled subgraph federated learning framework
• Leverages global graph structure dependencies
• No sharing of sensitive node features