Sheaf Graph Neural Networks via PAC-Bayes Spectral Optimization

2025-08-01

AI Summary

To address node feature collapse in graph neural networks (GNNs) on heterophilous graphs caused by oversmoothing, this paper proposes a cellular layer bundle—a novel message-passing architecture. The method integrates optimal transport enhancement, variance-reduced diffusion, and a new PAC-Bayes spectral regularizer to enable dynamic, topology-adaptive layer-structure learning, backed by verifiable stability guarantees. Unlike static or over-parameterized layer-bundle models, our approach supports end-to-end training and achieves linear-time inference complexity. Evaluated on nine homogeneous and heterophilous graph benchmarks, it significantly outperforms existing spectral methods and layer-bundle GNNs in semi-supervised node classification, achieving state-of-the-art performance. Moreover, it is the first to provide certified confidence intervals for unseen nodes—demonstrating superior generalization and robustness under distributional shift.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Over-smoothing in Graph Neural Networks (GNNs) causes collapse in distinct node features, particularly on heterophilic graphs where adjacent nodes often have dissimilar labels. Although sheaf neural networks partially mitigate this problem, they typically rely on static or heavily parameterized sheaf structures that hinder generalization and scalability. Existing sheaf-based models either predefine restriction maps or introduce excessive complexity, yet fail to provide rigorous stability guarantees. In this paper, we introduce a novel scheme called SGPC (Sheaf GNNs with PAC-Bayes Calibration), a unified architecture that combines cellular-sheaf message passing with several mechanisms, including optimal transport-based lifting, variance-reduced diffusion, and PAC-Bayes spectral regularization for robust semi-supervised node classification. We establish performance bounds theoretically and demonstrate that the resulting bound-aware objective can be achieved via end-to-end training in linear computational complexity. Experiments on nine homophilic and heterophilic benchmarks show that SGPC outperforms state-of-the-art spectral and sheaf-based GNNs while providing certified confidence intervals on unseen nodes.

Problem

• Address over-smoothing in GNNs on heterophilic graphs
• Overcome static sheaf structures limiting generalization
• Provide rigorous stability guarantees for sheaf-based models

Innovation

• PAC-Bayes spectral regularization for stability
• Optimal transport-based lifting for feature enhancement
• Variance-reduced diffusion for scalable training