Probing Graph Neural Network Activation Patterns Through Graph Topology

2026-02-24

AI Summary

This study investigates the relationship between activation patterns in graph neural networks (GNNs) and the curvature of graph topology, uncovering intrinsic mechanisms behind information propagation bottlenecks and model failure. Through large-scale activation analysis, it pioneers the use of graph curvature as a diagnostic tool to systematically examine the association between edge-wise activation extremes and curvature in graph Transformers. The findings reveal that global attention mechanisms induce a systematic “curvature shift,” challenging the conventional view that information flow is primarily governed by curvature extrema. Empirical evaluations on synthetic graphs, molecular datasets, and the Long Range Graph Benchmark demonstrate a significant increase in negative curvature regions, with high-magnitude activations not preferentially concentrated at curvature extrema—offering a novel perspective for understanding GNN behavior.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

Curvature notions on graphs provide a theoretical description of graph topology, highlighting bottlenecks and denser connected regions. Artifacts of the message passing paradigm in Graph Neural Networks, such as oversmoothing and oversquashing, have been attributed to these regions. However, it remains unclear how the topology of a graph interacts with the learned preferences of GNNs. Through Massive Activations, which correspond to extreme edge activation values in Graph Transformers, we probe this correspondence. Our findings on synthetic graphs and molecular benchmarks reveal that MAs do not preferentially concentrate on curvature extremes, despite their theoretical link to information flow. On the Long Range Graph Benchmark, we identify a systemic \textit{curvature shift}: global attention mechanisms exacerbate topological bottlenecks, drastically increasing the prevalence of negative curvature. Our work reframes curvature as a diagnostic probe for understanding when and why graph learning fails.

Problem

• Graph Neural Networks
• Graph Topology
• Curvature
• Activation Patterns
• Oversmoothing

Innovation

• Graph Neural Networks
• Graph Curvature
• Massive Activations
• Oversquashing
• Graph Transformers