An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regression

2025-04-25

AI Summary

Existing uncertainty quantification (UQ) research predominantly focuses on classification, leaving regression UQ lacking a rigorous axiomatic foundation and systematic evaluation framework. Method: We establish the first formal axiomatic system for regression UQ, unifying aleatoric, epistemic, and total uncertainty. We theoretically analyze fundamental limitations of two dominant uncertainty measures—differential entropy and variance—proving that neither can simultaneously satisfy key axioms under exponential-family predictive models. Building on this insight, we propose a unified evaluation paradigm grounded in exponential-family modeling, integrating information-theoretic principles with variance decomposition to derive principled validity criteria for regression uncertainty measures. Contribution/Results: This work fills a longstanding gap in the axiomatic study of regression UQ, providing both theoretical grounding—via formal axioms and impossibility results—and practical guidance—through a coherent, model-aware evaluation framework—thereby advancing both foundational understanding and real-world deployment of regression uncertainty estimation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Security and Privacy: Large-scale security measurements

Abstract

Uncertainty quantification (UQ) is crucial in machine learning, yet most (axiomatic) studies of uncertainty measures focus on classification, leaving a gap in regression settings with limited formal justification and evaluations. In this work, we introduce a set of axioms to rigorously assess measures of aleatoric, epistemic, and total uncertainty in supervised regression. By utilizing a predictive exponential family, we can generalize commonly used approaches for uncertainty representation and corresponding uncertainty measures. More specifically, we analyze the widely used entropy- and variance-based measures regarding limitations and challenges. Our findings provide a principled foundation for UQ in regression, offering theoretical insights and practical guidelines for reliable uncertainty assessment.

Problem

• Assess uncertainty measures in regression settings
• Evaluate entropy- and variance-based UQ limitations
• Establish axioms for aleatoric and epistemic uncertainty

Innovation

• Introduces axioms for uncertainty measures in regression
• Uses predictive exponential family for generalization
• Analyzes entropy- and variance-based measures limitations