Crowdsourcing the Frontier: Advancing Hybrid Physics-ML Climate Simulation via $50,000 Kaggle Competition

2025-11-25

AI Summary

Machine learning (ML) parameterizations for climate modeling often suffer from online instability and inconsistent performance when coupled to full-physics climate models. Method: This study leverages a $50,000 Kaggle competition to crowdsource surrogate models for subgrid-scale physical processes, using the ClimSim dataset. Multiple deep learning architectures are designed and rigorously evaluated via online coupling to an interactive climate model featuring comprehensive cloud microphysics. Contribution/Results: All top-performing competition models achieve long-term online stability. Expanding input variables significantly improves predictive accuracy. Several architectures attain state-of-the-art (SOTA) performance on key metrics—including zonal-mean bias and global root-mean-square error—while exhibiting strong offline–online consistency. This work constitutes the first systematic demonstration of the feasibility and reproducibility of the crowdsourcing paradigm for climate ML parameterization. It establishes a viable pathway toward high-resolution, computationally efficient, and reliable long-term climate prediction.

Technology Category

• Machine Learning: Auto ML and Hyperparameter Tuning
• Humans and AI: Human-in-the-loop Machine Learning
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Subgrid machine-learning (ML) parameterizations have the potential to introduce a new generation of climate models that incorporate the effects of higher-resolution physics without incurring the prohibitive computational cost associated with more explicit physics-based simulations. However, important issues, ranging from online instability to inconsistent online performance, have limited their operational use for long-term climate projections. To more rapidly drive progress in solving these issues, domain scientists and machine learning researchers opened up the offline aspect of this problem to the broader machine learning and data science community with the release of ClimSim, a NeurIPS Datasets and Benchmarks publication, and an associated Kaggle competition. This paper reports on the downstream results of the Kaggle competition by coupling emulators inspired by the winning teams' architectures to an interactive climate model (including full cloud microphysics, a regime historically prone to online instability) and systematically evaluating their online performance. Our results demonstrate that online stability in the low-resolution, real-geography setting is reproducible across multiple diverse architectures, which we consider a key milestone. All tested architectures exhibit strikingly similar offline and online biases, though their responses to architecture-agnostic design choices (e.g., expanding the list of input variables) can differ significantly. Multiple Kaggle-inspired architectures achieve state-of-the-art (SOTA) results on certain metrics such as zonal mean bias patterns and global RMSE, indicating that crowdsourcing the essence of the offline problem is one path to improving online performance in hybrid physics-AI climate simulation.

Problem

• Develop stable machine-learning subgrid parameterizations for climate models
• Address online instability in hybrid physics-ML climate simulations
• Improve long-term climate projection accuracy via crowdsourced solutions

Innovation

• Hybrid physics-ML parameterizations for climate models
• Crowdsourced Kaggle competition to solve offline issues
• Coupling winning architectures to interactive climate simulation