Uncertainty Modeling in Graph Neural Networks via Stochastic Differential Equations

2024-08-28

arXiv.org

AI Summary

Graph neural networks (GNNs) lack principled uncertainty quantification, and existing graph neural ordinary differential equations (GNODEs) cannot distinguish between epistemic and aleatoric uncertainty. Method: We propose Latent-variable Graph Neural Stochastic Differential Equations (LGNSDE), a Bayesian framework that jointly models both uncertainty types in latent space by integrating prior-posterior inference with Brownian-motion-driven stochastic dynamics. Contribution/Results: Theoretically, we establish for the first time the existence and uniqueness of solutions to graph-structured stochastic differential equations (SDEs), prove that latent-space variance bounds output uncertainty, and provide time-stability guarantees under input perturbations. Empirically, LGNSDE achieves state-of-the-art performance on out-of-distribution detection, noise-robust learning, and active learning—demonstrating significant improvements in reliability, interpretability, and generalizability of uncertainty estimation.

Technology Category

• Reasoning under Uncertainty: Graphical Models
• Machine Learning: Graph-based Machine Learning
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

We propose a novel Stochastic Differential Equation (SDE) framework to address the problem of learning uncertainty-aware representations for graph-structured data. While Graph Neural Ordinary Differential Equations (GNODEs) have shown promise in learning node representations, they lack the ability to quantify uncertainty. To address this, we introduce Latent Graph Neural Stochastic Differential Equations (LGNSDE), which enhance GNODE by embedding randomness through a Bayesian prior-posterior mechanism for epistemic uncertainty and Brownian motion for aleatoric uncertainty. By leveraging the existence and uniqueness of solutions to graph-based SDEs, we prove that the variance of the latent space bounds the variance of model outputs, thereby providing theoretically sensible guarantees for the uncertainty estimates. Furthermore, we show mathematically that LGNSDEs are robust to small perturbations in the input, maintaining stability over time. Empirical results across several benchmarks demonstrate that our framework is competitive in out-of-distribution detection, robustness to noise, and active learning, underscoring the ability of LGNSDEs to quantify uncertainty reliably.

Problem

• Uncertainty-aware graph data learning
• Enhancing GNODE with Bayesian mechanisms
• Proving stability and robustness in LGNSDEs

Innovation

• Stochastic Differential Equations framework
• Bayesian prior-posterior mechanism
• Brownian motion for uncertainty