Demystifying MPNNs: Message Passing as Merely Efficient Matrix Multiplication

2025-01-31

AI Summary

This work addresses the contradictory degradation in performance and lack of theoretical understanding of Graph Neural Networks (GNNs) on deep architectures, sparse graphs, and homophilous graphs. Methodologically, it introduces the first rigorous theoretical framework that formalizes *k*-layer message passing as matrix exponentiation over *k*-hop neighborhoods—unifying the characterization of information aggregation, cycle-induced interference, and structural-feature entanglement. By modeling gradient propagation, it identifies gradient degradation—not over-smoothing—as the primary bottleneck in deep GNNs, and establishes a structure-feature decoupling analytical paradigm. Contributions include: (i) the first matrix-power-based rigorous foundation for Message Passing Neural Networks (MPNNs), explaining their empirical success; and (ii) interpretable, principled guidelines for layer-depth selection, normalization design, and training on sparse graphs—substantially improving generalization and interpretability.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

While Graph Neural Networks (GNNs) have achieved remarkable success, their design largely relies on empirical intuition rather than theoretical understanding. In this paper, we present a comprehensive analysis of GNN behavior through three fundamental aspects: (1) we establish that extbf{$k$-layer} Message Passing Neural Networks efficiently aggregate extbf{$k$-hop} neighborhood information through iterative computation, (2) analyze how different loop structures influence neighborhood computation, and (3) examine behavior across structure-feature hybrid and structure-only tasks. For deeper GNNs, we demonstrate that gradient-related issues, rather than just over-smoothing, can significantly impact performance in sparse graphs. We also analyze how different normalization schemes affect model performance and how GNNs make predictions with uniform node features, providing a theoretical framework that bridges the gap between empirical success and theoretical understanding.

Problem

• Graph Neural Networks
• Theoretical Understanding
• Performance Gap

Innovation

• Graph Neural Networks
• Deeper GNNs
• Message Passing Neural Networks