Cooperative Sheaf Neural Networks

2025-07-01

AI Summary

Existing layer-wise diffusion models lack directional message propagation, preventing nodes from autonomously selecting information pathways and hindering collaborative communication. To address this, we propose a cellular-layer architecture tailored for directed graphs—the first such extension of cellular layers to directed settings—defining in-degree and out-degree Laplacian operators to endow nodes with “selective listening” capability. We further design a direction-aware message-passing mechanism and incorporate attention-based routing to dynamically model long-range dependencies. Our approach effectively mitigates over-smoothing and over-compression, achieving significant improvements in node classification accuracy on heterophilic graphs, outperforming state-of-the-art diffusion-based and collaborative GNNs. The core contribution lies in establishing a *directional information propagation paradigm* on directed graphs, enabling genuine node-level collaborative learning.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

Sheaf diffusion has recently emerged as a promising design pattern for graph representation learning due to its inherent ability to handle heterophilic data and avoid oversmoothing. Meanwhile, cooperative message passing has also been proposed as a way to enhance the flexibility of information diffusion by allowing nodes to independently choose whether to propagate/gather information from/to neighbors. A natural question ensues: is sheaf diffusion capable of exhibiting this cooperative behavior? Here, we provide a negative answer to this question. In particular, we show that existing sheaf diffusion methods fail to achieve cooperative behavior due to the lack of message directionality. To circumvent this limitation, we introduce the notion of cellular sheaves over directed graphs and characterize their in- and out-degree Laplacians. We leverage our construction to propose Cooperative Sheaf Neural Networks (CSNNs). Theoretically, we characterize the receptive field of CSNN and show it allows nodes to selectively attend (listen) to arbitrarily far nodes while ignoring all others in their path, potentially mitigating oversquashing. Our experiments show that CSNN presents overall better performance compared to prior art on sheaf diffusion as well as cooperative graph neural networks.

Problem

• Can sheaf diffusion achieve cooperative behavior in graphs?
• Existing sheaf diffusion lacks message directionality for cooperation.
• Propose Cooperative Sheaf Neural Networks to mitigate oversquashing.

Innovation

• Cellular sheaves over directed graphs
• In- and out-degree Laplacians
• Cooperative Sheaf Neural Networks