Hierarchical Testing of a Hybrid Machine Learning-Physics Global Atmosphere Model

2026-02-11

AI Summary

This study systematically evaluates the reliability and sources of bias in the hybrid machine learning–physics model NeuralGCM across multiple timescales and under out-of-distribution forcings, such as uniform warming. Through a unified experimental framework encompassing weather-scale forecasts, interannual variability driven by El Niño SST forcing, and idealized warming scenarios—augmented with cyclone tracking and teleconnection analyses—the work presents the first hierarchical, multi-scale assessment of a hybrid climate model under diverse forcing conditions. Results show that NeuralGCM closely reproduces key features of traditional physics-based models in cyclone propagation, teleconnection responses, and large-scale circulation changes under warming. However, it exhibits systematic biases, including overestimated cyclone spatial extent, phase errors in wave trains, and insufficient stratospheric response, thereby revealing both the promise and current limitations of hybrid modeling paradigms.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Machine learning (ML)-based models have demonstrated high skill and computational efficiency, often outperforming conventional physics-based models in weather and subseasonal predictions. While prior studies have assessed their fidelity in capturing synoptic-scale atmospheric dynamics, their performance across timescales and under out-of-distribution forcing, such as +3K or +4K uniform-warming forcings, and the sources of biases remain elusive, to establish the model reliability for Earth science. Here, we design three sets of experiments targeting synoptic-scale phenomena, interannual variability, and out-of-distribution uniform-warming forcings. We evaluate the Neural General Circulation Model (NeuralGCM), a hybrid model integrating a dynamical core with ML-based component, against observations and physics-based Earth system models (ESMs). At the synoptic scale, NeuralGCM captures the evolution and propagation of extratropical cyclones with performance comparable to ESMs. At the interannual scale, when forced by El Ni\~no-Southern Oscillation sea surface temperature (SST) anomalies, NeuralGCM successfully reproduces associated teleconnection patterns but exhibits deficiencies in capturing nonlinear response. Under out-of-distribution uniform-warming forcings, NeuralGCM simulates similar responses in global-average temperature and precipitation and reproduces large-scale tropospheric circulation features similar to those in ESMs. Notable weaknesses include overestimating the tracks and spatial extent of extratropical cyclones, biases in the teleconnected wave train triggered by tropical SST anomalies, and differences in upper-level warming and stratospheric circulation responses to SST warming compared to physics-based ESMs. The causes of these weaknesses were explored.

Problem

• hybrid ML-physics model
• atmospheric dynamics
• out-of-distribution forcing
• model bias
• climate reliability

Innovation

• hybrid ML-physics modeling
• hierarchical evaluation
• out-of-distribution forcing
• NeuralGCM
• atmospheric teleconnection