Trustworthy predictive distributions for rare events via diagnostic transport maps

2026-03-11

AI Summary

Existing forecasting systems often produce miscalibrated predictive distributions under high-dimensional, complex inputs, particularly yielding unreliable uncertainty estimates for rare or out-of-distribution events, which undermines expert trust in model outputs. This work proposes a covariate-dependent diagnostic transport mapping approach that dynamically recalibrates base predictive distributions through localized, interpretable probability–probability transformations. The method not only identifies local failure modes of the model—such as biases, dispersion errors, skewness distortions, and tail behavior inaccuracies—but also enables straightforward composition to generate well-calibrated predictive distributions. Evaluated on short-term tropical cyclone intensity forecasting, the approach significantly improves prediction performance for rare rapid intensification events over 24 hours, outperforming operational forecasts from the National Hurricane Center.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Forecast systems in science and technology are increasingly moving beyond point prediction toward methods that produce full predictive distributions of future outcomes y, conditional on high-dimensional and complex sequences of inputs x. However, even when forecast systems provide a full predictive distribution, the result is rarely calibrated with respect to all x and y. The estimated density can be especially unreliable in low-frequency or out-of-distribution regimes, where accurate uncertainty quantification and a means for human experts to verify results are most needed to establish trust in models. In this paper, we take an initial predictive distribution as given and treat it as a useful but potentially misspecified base model. WE then introduce diagnostic transport maps, covariate-dependent probability-to-probability maps that quantify how the base model's probabilities should be adjusted to better match the true conditional distribution of calibration data. At deployment, these maps provide the user with real-time local diagnostics that reveal where the model fails and how it fails (including bias, dispersion, skewness, and tail errors), while also producing a recalibrated predictive distribution through a simple composition with the base model. We apply diagnostic transport maps to short-term tropical cyclone intensity forecasting and show that an easy-to-fit parametric version identifies evolutionary modes associated with local miscalibration and improves the predictive performance for rare events, including 24-hour rapid intensity change, as compared to the operational forecasts of the National Hurricane Center.

Problem

• predictive distribution
• calibration
• rare events
• uncertainty quantification
• model trustworthiness

Innovation

• diagnostic transport maps
• predictive distribution calibration
• rare event forecasting
• covariate-dependent recalibration
• uncertainty quantification