Sheaf Neural Networks on SPD Manifolds: Second-Order Geometric Representation Learning

2026-04-22

AI Summary

Graph neural networks are inherently limited by the linear structure of Euclidean space, struggling to model covariant relationships among directional features, and standard message-passing lacks edge-specific transformation capabilities. This work proposes the first natively manifold-based layer-wise neural network operating directly on the symmetric positive definite (SPD) manifold. Leveraging the Lie group structure of SPD matrices, the method constructs layer operators that map directional inputs to full-rank matrices encoding local geometric information—without requiring projection into Euclidean space. Theoretically, SPD-valued layers are shown to be strictly more expressive than conventional vector-valued counterparts, capturing coherent global structures that the latter cannot represent. Empirically, the approach achieves state-of-the-art performance on six out of seven MoleculeNet benchmarks and demonstrates exceptional robustness with increasing network depth.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Other Foundations of Computer Vision
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Social Networks and Social Media: Social media analysis through the lenses of networks

Abstract

Graph neural networks face two fundamental challenges rooted in the linear structure of Euclidean vector spaces: (1) Current architectures represent geometry through vectors (directions, gradients), yet many tasks require matrix-valued representations that capture relationships between directions-such as how atomic orientations covary in a molecule. These second-order representations are naturally captured by points on the symmetric positive definite matrices (SPD) manifold; (2) Standard message passing applies shared transformations across edges. Sheaf neural networks address this via edge-specific transformations, but existing formulations remain confined to vector spaces and therefore cannot propagate matrix-valued features. We address both challenges by developing the first sheaf neural network operates natively on the SPD manifold. Our key insight is that the SPD manifold admits a Lie group structure, enabling well-posed analogs of sheaf operators without projecting to Euclidean space. Theoretically, we prove that SPD-valued sheaves are strictly more expressive than Euclidean sheaves: they admit consistent configurations (global sections) that vector-valued sheaves cannot represent, directly translating to richer learned representations. Empirically, our sheaf convolution transforms effectively rank-1 directional inputs into full-rank matrices encoding local geometric structure. Our dual-stream architecture achieves SOTA on 6/7 MoleculeNet benchmarks, with the sheaf framework providing consistent depth robustness.

Problem

• Sheaf Neural Networks
• SPD Manifolds
• Second-Order Representation
• Geometric Deep Learning
• Matrix-Valued Features

Innovation

• Sheaf Neural Networks
• SPD Manifold
• Second-Order Representation
• Lie Group Structure
• Geometric Deep Learning