Uncertainty for Active Learning on Graphs

2024-05-02

International Conference on Machine Learning

AI Summary

This work addresses the significant performance degradation of uncertainty sampling for node classification on graph-structured data compared to i.i.d. settings, systematically uncovering the failure mechanisms of existing uncertainty estimation methods under graph topology. We propose the first provably optimal Bayesian uncertainty ground truth benchmark, derived from a principled graph generative process model. To enable practical deployment, we design an efficient approximation algorithm that tightly integrates Bayesian inference with graph neural networks, achieving high-fidelity estimation of the true uncertainty. Extensive experiments on multiple real-world graph datasets demonstrate that our method consistently outperforms state-of-the-art uncertainty estimation techniques, substantially improving active learning query quality. This work bridges a critical theoretical gap in uncertainty quantification and active learning for graph-structured data, establishing a verifiable benchmark and a practical framework for Bayesian active learning on graphs.

Technology Category

• Reasoning under Uncertainty: Graphical Models
• Machine Learning: Calibration & Uncertainty Quantification
• Planning, Routing, and Scheduling: Planning under Uncertainty

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Uncertainty Sampling is an Active Learning strategy that aims to improve the data efficiency of machine learning models by iteratively acquiring labels of data points with the highest uncertainty. While it has proven effective for independent data its applicability to graphs remains under-explored. We propose the first extensive study of Uncertainty Sampling for node classification: (1) We benchmark Uncertainty Sampling beyond predictive uncertainty and highlight a significant performance gap to other Active Learning strategies. (2) We develop ground-truth Bayesian uncertainty estimates in terms of the data generating process and prove their effectiveness in guiding Uncertainty Sampling toward optimal queries. We confirm our results on synthetic data and design an approximate approach that consistently outperforms other uncertainty estimators on real datasets. (3) Based on this analysis, we relate pitfalls in modeling uncertainty to existing methods. Our analysis enables and informs the development of principled uncertainty estimation on graphs.

Problem

• Active Learning on Graphs
• Uncertainty Sampling for Node Classification
• Bayesian Uncertainty Estimates

Innovation

• Uncertainty Sampling for graphs
• Bayesian uncertainty estimates
• Approximate approach outperforms estimators