Leveraging data-driven weather models for improving numerical weather prediction skill through large-scale spectral nudging

2024-07-08

arXiv.org

AI Summary

Pure data-driven weather models suffer from low resolution, incomplete variable coverage, and poor extreme-weather representation in medium- to long-range forecasting. To address these limitations, we propose a physics-AI hybrid forecasting framework that couples the Canadian Global Environmental Multiscale (GEM) numerical model with the GraphCast large AI model. Our key innovation is the first implementation of large-scale dynamical nudging in spectral space: GraphCast’s predicted large-scale circulation fields are used to constrain and dynamically adjust GEM’s spectral coefficients in real time. This preserves GEM’s high-fidelity simulation of small-scale physical processes—such as tropical cyclone structure—while substantially improving forecast accuracy. Experiments demonstrate a 15–20% reduction in tropical cyclone track error, enhanced intensity forecast stability, and full output of all operational meteorological variables. The framework overcomes the fundamental trade-offs among resolution, variable completeness, and physical consistency inherent in purely AI-based models.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Reasoning under Uncertainty: Graphical Models
• Computer Vision: Remote Sensing / Geospatial AI

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Operational meteorological forecasting has long relied on physics-based numerical weather prediction (NWP) models. Recently, this landscape is facing disruption by the advent of data-driven artificial intelligence (AI)-based weather models, which offer tremendous computational performance and competitive forecasting skill. However, data-driven models for medium-range forecasting generally suffer from major limitations, including low effective resolution and a narrow range of predicted variables. This study illustrates the relative strengths and weaknesses of these competing paradigms using the GEM (Global Environmental Multiscale) and GraphCast models to represent physics-based and AI-based approaches, respectively. By analyzing global predictions from these two models against observations and analyses in both physical and spectral spaces, this study demonstrates that GraphCast-predicted large scales outperform GEM, particularly for longer lead times. Building on this insight, a hybrid NWP-AI system is proposed, wherein GEM-predicted large-scale state variables are spectrally nudged toward GraphCast predictions, while allowing GEM to freely generate fine-scale details critical for weather extremes. Results indicate that this hybrid approach is capable of leveraging the strengths of GraphCast to enhance the prediction skill of the GEM model. Importantly, trajectories of tropical cyclones are predicted with enhanced accuracy without significant changes in intensity. Furthermore, this new hybrid system ensures that meteorologists have access to a complete set of forecast variables, including those relevant for high-impact weather events.

Problem

• Improve medium-range weather prediction accuracy using AI and physics-based models
• Address low resolution and limited variables in data-driven weather models
• Combine AI and NWP strengths for better large-scale and local forecasts

Innovation

• Hybrid NWP-AI system combining GEM and GraphCast
• Spectral nudging for large-scale weather predictions
• Enhanced tropical cyclone trajectory forecasting accuracy