Uncertainty quantification via conformal prediction in data assimilation

2026-06-25

AI Summary

This study addresses the insufficient quantification of uncertainty in numerical weather prediction by introducing, for the first time, multiple conformal prediction (CP) methods—including standard CP, normalized CP, and conformal quantile regression—into a one-dimensional shallow-water model data assimilation framework. These methods are integrated with the ensemble Kalman filter to produce prediction intervals endowed with finite-sample theoretical guarantees. Systematic evaluation using metrics such as average coverage, interval width, upper and lower tail miss rates, and interval score demonstrates that CP effectively characterizes forecast uncertainty and complements traditional ensemble-based approaches within the data assimilation cycle. The work establishes a novel paradigm and empirical foundation for uncertainty quantification that synergistically combines machine learning with physics-driven modeling.

Abstract

Quantifying the evolution of uncertainty is critical to both probabilistic forecasting and data assimilation in numerical weather prediction. In this study, we investigate the applicability of conformal prediction (CP), a recent machine learning (ML) method, to quantify uncertainty in a controlled, idealized setting. We use the one dimensional modified shallow water model, designed to mimic the convective process. CP provides a set of possible outcomes with a chosen confidence level. Here, we compare and evaluate the average empirical coverage, the average interval length, miss low, miss high and average interval score loss (AISL) for three variants of CP, namely a) Standard CP, b) Normalized CP and c) Conformalized Quantile Regression. We further compare these CP-based uncertainty estimates with traditional ensemble-based measures such as standard deviation intervals and ensemble spread. In addition, we investigate the integration of CP-derived uncertainty within the data assimilation cycle through CP perturbations. Our results highlight the strengths and limitations of each approach, providing insight into the effectiveness of CP to complement common ensemble-based uncertainty quantification in simplified atmospheric models.

Problem

• uncertainty quantification
• conformal prediction
• data assimilation
• numerical weather prediction
• ensemble methods

Innovation

• Conformal Prediction
• Uncertainty Quantification
• Data Assimilation
• Ensemble Methods
• Shallow Water Model