Making Tunable Parameters State-Dependent in Weather and Climate Models with Reinforcement Learning

2026-01-07

arXiv.org

AI Summary

This study addresses the limitations of traditional weather and climate models, which rely on fixed parameterization schemes that struggle to adapt to dynamically evolving physical states and often introduce systematic biases. To overcome this, the work proposes a novel reinforcement learning–based adaptive parameterization framework that is both state-dependent and region-aware, supporting both single-agent and federated multi-agent collaborative optimization. Leveraging advanced algorithms—including TQC, DDPG, and TD3—the framework enables online parameter adjustment within idealized climate models such as SCBC, RCE, and EBM. Experimental results demonstrate significant reductions in RMSE for temperature profiles and meridional radiative flux biases across tropical and mid-latitude regions, achieving physically consistent and stable performance improvements.

Technology Category

• Machine Learning: Reinforcement Learning
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Weather and climate models rely on parametrisations to represent unresolved sub-grid processes. Traditional schemes rely on fixed coefficients that are weakly constrained and tuned offline, contributing to persistent biases that limit their ability to adapt to the underlying physics. This study presents a framework that learns components of parametrisation schemes online as a function of the evolving model state using reinforcement learning (RL) and evaluates the resulting RL-driven parameter updates across a hierarchy of idealised testbeds spanning a simple climate bias correction (SCBC), a radiative-convective equilibrium (RCE), and a zonal mean energy balance model (EBM) with both single-agent and federated multi-agent settings. Across nine RL algorithms, Truncated Quantile Critics (TQC), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed DDPG (TD3) achieved the highest skill and the most stable convergence across configurations, with performance assessed against a static baseline using area-weighted RMSE, temperature profile and pressure-level diagnostics. For the EBM, single-agent RL outperformed static parameter tuning with the strongest gains in tropical and mid-latitude bands, while federated RL on multi-agent setups enabled geographically specialised control and faster convergence, with a six-agent DDPG configuration using frequent aggregation yielding the lowest area-weighted RMSE across the tropics and mid-latitudes. The learnt corrections were also physically meaningful as agents modulated EBM radiative parameters to reduce meridional biases, adjusted RCE lapse rates to match vertical temperature errors, and stabilised SCBC heating increments to limit drift. Overall, results highlight RL to deliver skilful state-dependent, and regime-aware parametrisations, offering a scalable pathway for online learning within numerical models.

Problem

• parameterization
• state-dependent
• climate models
• weather models
• model bias

Innovation

• reinforcement learning
• state-dependent parametrization
• climate modeling
• federated multi-agent
• online learning