On the QUEST for Uncertainty Quantification via Highest Density Regions

2026-06-17

AI Summary

This work addresses the limitations of existing uncertainty quantification methods in regression tasks, which predominantly rely on pointwise predictive risk and struggle to characterize uncertainty under non-conditional-expectation objectives. The authors propose QUEST, a novel framework that introduces highest density regions (HDRs) into uncertainty quantification, measuring uncertainty via the Lebesgue measure of the HDR’s support set and incorporating a robustness parameter α to capture the concentration of probability mass around distributional modes. This metric satisfies key axioms such as monotonicity and translation invariance, overcoming fundamental shortcomings of conventional scoring rules. Empirical evaluations on selective prediction benchmarks demonstrate that QUEST significantly outperforms standard uncertainty measures—such as predictive variance and differential entropy—in capturing both aleatoric and epistemic uncertainty.

Abstract

Uncertainty quantification (UQ) is essential for reliable decision-making in safety-critical applications in probabilistic machine learning. For regression problems, dominant scalar UQ approaches - notably, those based on proper scoring rules - measure uncertainty via pointwise predictive risk. This can lead to counterintuitive results when the target statistic is not the conditional expectation. We propose an alternative framework, in which uncertainty is characterised by the volume of the most probable subset of a distribution's support. QUEST (Quantifying Uncertainty via highest dEnSiTy regions) is a novel approach to UQ based on the concentration of Lebesgue measure at a distribution's peak(s), evaluated at one or more values of a robustness parameter $α$. We establish connections between our measures and classical statistics from information theory and economics. We show that, unlike popular alternatives based on proper scoring rules, QUEST measures of epistemic and aleatoric uncertainty satisfy a set of axioms adapted from the UQ literature, including monotonicity under distributional spread and invariance to location shifts. Selective prediction benchmarks confirm that QUEST performs favourably against standard measures such as variance and differential entropy.

Problem

• Uncertainty Quantification
• Regression
• Proper Scoring Rules
• Epistemic Uncertainty
• Aleatoric Uncertainty

Innovation

• uncertainty quantification
• highest density regions
• proper scoring rules
• epistemic uncertainty
• Lebesgue measure concentration