On the Predictive Skill of Artificial Intelligence-based Weather Models for Extreme Events using Uncertainty Quantification

2025-11-21

AI Summary

Accurate prediction of extreme weather events remains a core challenge for AI-based meteorological forecasting. While state-of-the-art deterministic models (e.g., GraphCast, FuXi) achieve skill comparable to numerical weather prediction (NWP), they lack uncertainty quantification and fail to adequately capture extremes. To address this, we propose a flow-dependent initial perturbation method that extends deterministic AI models into a 50-member ensemble forecasting system, integrating three complementary perturbation strategies: Gaussian noise, hemispheric-center Bred vectors, and Huge Ensembles. Experiments demonstrate substantial improvements in probabilistic skill and spread reliability for temperature extremes, with performance approaching that of conventional NWP ensembles in case studies of the Pakistan flood and China heatwave; precipitation forecasts remain less skillful. This work provides the first systematic validation of flow-dependent perturbations for uncertainty quantification in AI weather models, establishing a novel paradigm for developing trustworthy, AI-driven early warning systems.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web search models and ranking

Abstract

Accurate prediction of extreme weather events remains a major challenge for artificial intelligence based weather prediction systems. While deterministic models such as FuXi, GraphCast, and SFNO have achieved competitive forecast skill relative to numerical weather prediction, their ability to represent uncertainty and capture extremes is still limited. This study investigates how state of the art deterministic artificial intelligence based models respond to initial-condition perturbations and evaluates the resulting ensembles in forecasting extremes. Using three perturbation strategies (Gaussian noise, Hemispheric Centered Bred Vectors, and Huge Ensembles), we generate 50 member ensembles for two major events in August 2022: the Pakistan floods and the China heatwave. Ensemble skill is assessed against ERA5 and compared with IFS ENS and the probabilistic AIFSENS model using deterministic and probabilistic metrics. Results show that flow dependent perturbations produce the most realistic ensemble spread and highest probabilistic skill, narrowing but not closing the performance gap with numerical weather prediction ensembles. Across variables, artificial intelligence based weather models capture temperature extremes more effectively than precipitation. These findings demonstrate that input perturbations can extend deterministic models toward probabilistic forecasting, paving the way for approaches that combine flow dependent perturbations with generative or latent-space uncertainty modeling for reliable artificial intelligence-driven early warning systems.

Problem

• Evaluating AI weather models' predictive skill for extreme events using uncertainty quantification
• Assessing ensemble forecasting performance for floods and heatwaves with perturbation strategies
• Comparing AI and numerical weather prediction models' capability in capturing temperature and precipitation extremes

Innovation

• Using flow-dependent perturbations for ensemble generation
• Evaluating AI models on extreme temperature and precipitation
• Combining perturbations with generative uncertainty modeling