S$^3$GNN: Efficient Global Mixing and Local Message Passing for Long-Range Graph Learning

2026-05-22

AI Summary

This work addresses the oversquashing problem that hinders message-passing neural networks in modeling long-range dependencies. The authors propose a lightweight global mixing module that effectively alleviates information bottlenecks by integrating local message passing with low-complexity spectral filtering, without requiring strong theoretical assumptions. The method incorporates standard stability constraints to ensure both robustness in feature transformation and computational efficiency. Experimental results demonstrate that the proposed approach reduces model error by up to an order of magnitude while cutting parameter count by as much as 50% across diverse tasks, including long-range graph learning, knowledge graph question answering, and mesh-based fluid dynamics simulations.

Abstract

Message-passing neural networks (MPNNs) often suffer from an information bottleneck when capturing long-range dependencies, leading to the oversquashing (OSQ) phenomenon. Alongside spatial connectivity enrichment (e.g., rewiring), recent studies have shown that spectral filtering can yield strong long-range learning outcomes, as spectral operators enable global information mixing that alleviates OSQ. These approaches achieve this either by stabilizing the Jacobian energies in deep propagation or by guaranteeing OSQ mitigation under strong theoretical assumptions. We revisit these conclusions and show that the associated Jacobian sensitivity lower bound is generally difficult to achieve in practice. We then propose S$^3$GNN, which mitigates OSQ without such restrictive assumptions by lightweightly reintroducing omitted components with substantially lower computational complexity, while standard stability constraints on feature transformations remain effective under our new dynamics. Extensive experiments across diverse domains (e.g., long-range benchmarks, KGQA, and mesh-based fluid dynamics) demonstrate that S$^3$GNN achieves up to an order-of-magnitude error reduction with up to 50\% fewer parameters. Our code can be found in https://github.com/EEthanShi/S3-GNN.git.

Problem

• oversquashing
• long-range dependencies
• message-passing neural networks
• graph learning
• information bottleneck

Innovation

• oversquashing
• spectral filtering
• message-passing neural networks
• global mixing
• graph neural networks