Local Virtual Nodes for Alleviating Over-Squashing in Graph Neural Networks

2025-08-28

AI Summary

Graph Neural Networks (GNNs) suffer from over-squashing when modeling long-range dependencies: during message passing, information from extensive neighborhoods is forced into fixed-dimensional node representations, creating an information bottleneck. While existing approaches—such as graph rewiring or global virtual nodes—alleviate over-squashing, they distort the original graph’s topology and undermine domain-specific structural knowledge. To address this, we propose Local Virtual Nodes (LVN): leveraging node centrality to identify critical bottleneck regions, LVN injects learnable virtual nodes *within local subgraphs* and shares their embeddings to facilitate information propagation among distant nodes—*without altering the global graph structure*. Extensive experiments on multiple graph and node classification benchmarks demonstrate that LVN consistently improves performance, validating its effectiveness in mitigating over-squashing while preserving domain-structured topology and enhancing local connectivity.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Search and Optimization: Local Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

Over-squashing is a challenge in training graph neural networks for tasks involving long-range dependencies. In such tasks, a GNN's receptive field should be large enough to enable communication between distant nodes. However, gathering information from a wide range of neighborhoods and squashing its content into fixed-size node representations makes message-passing vulnerable to bottlenecks. Graph rewiring and adding virtual nodes are commonly studied remedies that create additional pathways around bottlenecks to mitigate over-squashing. However, these techniques alter the input graph's global topology and disrupt the domain knowledge encoded in the original graph structure, both of which could be essential to specific tasks and domains. This study presents Local Virtual Nodes (LVN) with trainable embeddings to alleviate the effects of over-squashing without significantly corrupting the global structure of the input graph. The position of the LVNs is determined by the node centrality, which indicates the existence of potential bottlenecks. Thus, the proposed approach aims to improve the connectivity in the regions with likely bottlenecks. Furthermore, trainable LVN embeddings shared across selected central regions facilitate communication between distant nodes without adding more layers. Extensive experiments on benchmark datasets demonstrate that LVNs can enhance structural connectivity and significantly improve performance on graph and node classification tasks. The code can be found at https://github.com/ALLab-Boun/LVN/}{https://github.com/ALLab-Boun/LVN/.

Problem

• Addresses over-squashing in GNNs for long-range dependencies
• Mitigates bottlenecks without altering global graph topology
• Uses local virtual nodes to improve connectivity in central regions

Innovation

• Local Virtual Nodes with trainable embeddings
• Node centrality determines LVN placement
• Shared LVN embeddings improve connectivity