Exploring and Improving Initialization for Deep Graph Neural Networks: A Signal Propagation Perspective

2025-06-20

AI Summary

Deep graph neural networks (GNNs) often suffer from signal degradation, leading to performance deterioration with increasing depth. Method: Motivated by signal propagation analysis, we propose the Graph Embedding Variation (GEV) metric—the first quantitative measure to expose the inherent limitation of existing initialization schemes in jointly regulating forward/backward propagation and GEV. Based on this insight, we introduce SPoGInit, a systematic initialization framework integrating signal propagation theory, variance-aware weight optimization, GCN-specific design, and graph-structure-aware adaptation. Contribution/Results: Extensive experiments demonstrate that SPoGInit significantly enhances training stability and convergence accuracy for deep GNNs. It consistently outperforms standard baselines—including Xavier and He initializations—across diverse tasks and architectures (e.g., GCN, GAT, GIN), enabling sustained performance gains as network depth increases.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Graph Neural Networks (GNNs) often suffer from performance degradation as the network depth increases. This paper addresses this issue by introducing initialization methods that enhance signal propagation (SP) within GNNs. We propose three key metrics for effective SP in GNNs: forward propagation, backward propagation, and graph embedding variation (GEV). While the first two metrics derive from classical SP theory, the third is specifically designed for GNNs. We theoretically demonstrate that a broad range of commonly used initialization methods for GNNs, which exhibit performance degradation with increasing depth, fail to control these three metrics simultaneously. To deal with this limitation, a direct exploitation of the SP analysis--searching for weight initialization variances that optimize the three metrics--is shown to significantly enhance the SP in deep GCNs. This approach is called Signal Propagation on Graph-guided Initialization (SPoGInit). Our experiments demonstrate that SPoGInit outperforms commonly used initialization methods on various tasks and architectures. Notably, SPoGInit enables performance improvements as GNNs deepen, which represents a significant advancement in addressing depth-related challenges and highlights the validity and effectiveness of the SP analysis framework.

Problem

• Addressing performance degradation in deep Graph Neural Networks
• Improving signal propagation via specialized initialization methods
• Controlling forward, backward propagation and graph embedding variation

Innovation

• Introduces SPoGInit for deep GNN initialization
• Optimizes forward, backward propagation, and GEV metrics
• Enhances signal propagation in deep graph networks