Uncertainty Quantification for Regression using Proper Scoring Rules

2025-09-30

AI Summary

This paper addresses the lack of a unified theoretical framework for uncertainty quantification (UQ) in regression tasks. We systematically introduce proper scoring rules—including the Continuous Ranked Probability Score (CRPS), logarithmic score, and quadratic score—to establish a principled, unified UQ framework grounded in the principle of propriety. Our method enables closed-form analytical uncertainty estimation and model-ensemble-based inference, naturally decomposing total uncertainty into aleatoric and epistemic components. Unlike conventional variance- or differential-entropy-based approaches, our framework ensures theoretical rigor while enhancing interpretability, providing a coherent metric system and principled selection criteria for regression UQ. Extensive experiments on synthetic and real-world datasets validate its effectiveness: it successfully reproduces and unifies major existing UQ metrics, significantly improving the reliability and practical utility of uncertainty assessment.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Quantifying uncertainty of machine learning model predictions is essential for reliable decision-making, especially in safety-critical applications. Recently, uncertainty quantification (UQ) theory has advanced significantly, building on a firm basis of learning with proper scoring rules. However, these advances were focused on classification, while extending these ideas to regression remains challenging. In this work, we introduce a unified UQ framework for regression based on proper scoring rules, such as CRPS, logarithmic, squared error, and quadratic scores. We derive closed-form expressions for the resulting uncertainty measures under practical parametric assumptions and show how to estimate them using ensembles of models. In particular, the derived uncertainty measures naturally decompose into aleatoric and epistemic components. The framework recovers popular regression UQ measures based on predictive variance and differential entropy. Our broad evaluation on synthetic and real-world regression datasets provides guidance for selecting reliable UQ measures.

Problem

• Extends uncertainty quantification from classification to regression problems
• Develops unified framework using proper scoring rules for regression
• Decomposes uncertainty into aleatoric and epistemic components

Innovation

• Uses proper scoring rules for regression uncertainty
• Derives closed-form expressions under parametric assumptions
• Estimates uncertainty using ensembles of models