Multidimensional Uncertainty Quantification via Optimal Transport

2025-09-26

AI Summary

Existing uncertainty quantification (UQ) methods predominantly rely on single-scalar metrics, limiting their ability to comprehensively characterize predictive reliability—especially in capturing diverse failure modes. To address this, we propose VecUQ-OT, a framework that constructs high-dimensional uncertainty vectors from complementary heterogeneous sources (e.g., ensemble-based and density-estimation-based epistemic uncertainties) and establishes a Monge–Kantorovich order via entropy-regularized optimal transport, enabling non-additive, fully ordered, unified UQ. Crucially, VecUQ-OT generalizes to out-of-distribution (OOD) data without model retraining. It significantly enhances robustness across downstream tasks—including selective prediction, misclassification detection, and OOD identification. Extensive experiments on synthetic, image, and text benchmarks demonstrate consistent superiority over scalar-metric baselines.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Most uncertainty quantification (UQ) approaches provide a single scalar value as a measure of model reliability. However, different uncertainty measures could provide complementary information on the prediction confidence. Even measures targeting the same type of uncertainty (e.g., ensemble-based and density-based measures of epistemic uncertainty) may capture different failure modes. We take a multidimensional view on UQ by stacking complementary UQ measures into a vector. Such vectors are assigned with Monge-Kantorovich ranks produced by an optimal-transport-based ordering method. The prediction is then deemed more uncertain than the other if it has a higher rank. The resulting VecUQ-OT algorithm uses entropy-regularized optimal transport. The transport map is learned on vectors of scores from in-distribution data and, by design, applies to unseen inputs, including out-of-distribution cases, without retraining. Our framework supports flexible non-additive uncertainty fusion (including aleatoric and epistemic components). It yields a robust ordering for downstream tasks such as selective prediction, misclassification detection, out-of-distribution detection, and selective generation. Across synthetic, image, and text data, VecUQ-OT shows high efficiency even when individual measures fail. The code for the method is available at: https://github.com/stat-ml/multidimensional_uncertainty.

Problem

• Combining multiple uncertainty measures into multidimensional vectors
• Ranking predictions using optimal transport-based ordering method
• Enabling robust uncertainty quantification for various downstream tasks

Innovation

• Stacks complementary uncertainty measures into vectors
• Uses optimal transport for ranking uncertainty vectors
• Applies learned transport map to unseen data without retraining