CLEAR: Calibrated Learning for Epistemic and Aleatoric Risk

2025-07-10

AI Summary

Existing regression uncertainty quantification methods typically model aleatoric (measurement noise) or epistemic (model/data limitations) uncertainty in isolation, failing to balance both effectively. This work proposes CLEAR, a novel dual-parameter calibration framework that jointly calibrates quantile regression—capturing aleatoric uncertainty—and Prediction Confidence Sets (PCS)—capturing epistemic uncertainty—for simultaneous optimization of conditional coverage error. CLEAR is model-agnostic, requiring no modifications to the underlying predictor and compatible with diverse uncertainty estimators. Evaluated on 17 real-world datasets, CLEAR reduces average prediction interval width by 28.2% under high-epistemic-uncertainty conditions and by 17.4% under high-aleatoric-uncertainty conditions, relative to single-calibration baselines—while strictly maintaining nominal coverage guarantees. This yields substantial improvements in both predictive accuracy and reliability of uncertainty intervals.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Accurate uncertainty quantification is critical for reliable predictive modeling, especially in regression tasks. Existing methods typically address either aleatoric uncertainty from measurement noise or epistemic uncertainty from limited data, but not necessarily both in a balanced way. We propose CLEAR, a calibration method with two distinct parameters, $γ_1$ and $γ_2$, to combine the two uncertainty components for improved conditional coverage. CLEAR is compatible with any pair of aleatoric and epistemic estimators; we show how it can be used with (i) quantile regression for aleatoric uncertainty and (ii) ensembles drawn from the Predictability-Computability-Stability (PCS) framework for epistemic uncertainty. Across 17 diverse real-world datasets, CLEAR achieves an average improvement of 28.2% and 17.4% in the interval width compared to the two individually calibrated baselines while maintaining nominal coverage. This improvement can be particularly evident in scenarios dominated by either high epistemic or high aleatoric uncertainty.

Problem

• Balances aleatoric and epistemic uncertainty quantification
• Improves conditional coverage in predictive modeling
• Compatible with diverse uncertainty estimation methods

Innovation

• Combines aleatoric and epistemic uncertainty calibration
• Uses quantile regression for aleatoric uncertainty
• Applies PCS framework ensembles for epistemic uncertainty