Learning Latent Graph Structures and their Uncertainty

2024-05-30

arXiv.org

AI Summary

In graph neural networks (GNNs), the underlying graph structure is often unknown, and optimizing solely for node prediction loss can induce structural learning bias. Method: This paper proposes a novel framework for jointly learning both the latent graph structure and its uncertainty. Theoretically, we establish—for the first time—that a loss function tailored to stochastic outputs simultaneously ensures optimal learning of the adjacency matrix distribution and predictive performance. Methodologically, we design a differentiable, variational inference–based stochastic graph sampling mechanism, enabling end-to-end joint optimization of graph structure learning, uncertainty quantification, and GNN training. Results: Extensive experiments on multiple benchmark datasets demonstrate significant improvements in prediction accuracy, alongside well-calibrated uncertainty estimates over the inferred graph structure—achieving both theoretical rigor and empirical effectiveness.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Graph-based Machine Learning
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Within a prediction task, Graph Neural Networks (GNNs) use relational information as an inductive bias to enhance the model's accuracy. As task-relevant relations might be unknown, graph structure learning approaches have been proposed to learn them while solving the downstream prediction task. In this paper, we demonstrate that minimization of a point-prediction loss function, e.g., the mean absolute error, does not guarantee proper learning of the latent relational information and its associated uncertainty. Conversely, we prove that a suitable loss function on the stochastic model outputs simultaneously grants (i) the unknown adjacency matrix latent distribution and (ii) optimal performance on the prediction task. Finally, we propose a sampling-based method that solves this joint learning task. Empirical results validate our theoretical claims and demonstrate the effectiveness of the proposed approach.

Problem

• Learning latent graph structures from data without direct training signals
• Addressing uncertainty in learned graph topologies for better predictions
• Proposing a method to jointly learn graph distributions and target predictions

Innovation

• Learning latent graph structures from data
• Using stochastic model outputs for uncertainty
• Sampling-based method for joint learning