On the optimal prediction of extreme events

2026-06-24

AI Summary

This study addresses the challenge of predicting extreme upper-tail values of a response variable $Y$ given covariates $X$ under severe data scarcity. Focusing on $(Y,X)$ pairs with multivariate regularly varying distributions, the work proposes a class of asymptotically optimal positively homogeneous predictors. By linking predictive accuracy to the tail dependence structure, the problem is reformulated as a variational problem over the angular measure, and a general solution is derived for a broad class of models: the predictor is determined by non-extreme conditional quantiles of a tilted distribution induced by the angular measure. The estimator is theoretically shown to be consistent over a wide class of angular measures, and empirical results demonstrate its near-oracle performance, with strong results in challenging applications such as extreme solar flare forecasting.

Abstract

The prediction of the extremely large values of a response variable $Y$ in terms of a vector of covariates $X=(X_i)_{i=1}^d$ is a fundamental problem arising in many scientific and engineering domains. The scarcity of data in the extremes makes the optimal solution of this problem of particular importance. The optimal predictors of such events can be explicitly characterized in just a few cases and it is of fundamental practical and theoretical interest to develop optimal estimators over large classes of models and predictors. In this work, the focus is on the case where $(Y,X)$ have a multivariate regularly varying distribution and one seeks an optimal predictor expressed as a positive homogeneous function $h(X)$ of the covariates. The asymptotic prediction precision in this setting coincides with the tail-dependence coefficient $λ(Y,h(X))$ and it can be expressed as an integral functional of the associated angular measure of $(Y,X)$. Thus, finding asymptotically optimal homogeneous predictors amounts to solving a variational problem. We obtain a general solution to this problem, which is expressed in terms of a non-extreme conditional quantile of a tilted distribution derived from the angular measure. This leads to a general inference methodology for the optimal predictors in the peaks-over-threshold framework form extreme value theory. We establish the universal consistency for these estimators over large classes of angular measures. A general-purpose implementation of the resulting inference procedure is shown to work remarkably well against optimal oracle estimators, as well as in the challenging problem of extreme solar flare prediction.

Problem

• extreme events
• optimal prediction
• multivariate regularly varying
• tail-dependence coefficient
• homogeneous predictors

Innovation

• extreme value theory
• tail dependence
• angular measure
• homogeneous predictor
• peaks-over-threshold