CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer

2025-02-27

AI Summary

Subseasonal-to-seasonal (S2S) climate forecasting remains highly challenging due to the intrinsic chaos of the climate system; moreover, prevailing data-driven models often flatten spherical meteorological fields onto planar grids, introducing substantial geometric distortion. To address this, we propose the geometry-aware Circular Transformer (CirT), the first model to explicitly encode spherical periodicity via a novel latitude-wise circular tiling strategy and to integrate Fourier-transformed positional encodings into self-attention—thereby jointly modeling spherical spatial periodicity and long-range dependencies. CirT ensures geometrically consistent representation learning over spherical domains. Evaluated on the ERA5 reanalysis dataset, CirT achieves state-of-the-art performance in forecast accuracy, spatial coherence, and temporal stability—outperforming leading baselines including PanguWeather, GraphCast, and the operational ECMWF Integrated Forecasting System.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Computer Vision: Remote Sensing / Geospatial AI

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Accurate Subseasonal-to-Seasonal (S2S) climate forecasting is pivotal for decision-making including agriculture planning and disaster preparedness but is known to be challenging due to its chaotic nature. Although recent data-driven models have shown promising results, their performance is limited by inadequate consideration of geometric inductive biases. Usually, they treat the spherical weather data as planar images, resulting in an inaccurate representation of locations and spatial relations. In this work, we propose the geometric-inspired Circular Transformer (CirT) to model the cyclic characteristic of the graticule, consisting of two key designs: (1) Decomposing the weather data by latitude into circular patches that serve as input tokens to the Transformer; (2) Leveraging Fourier transform in self-attention to capture the global information and model the spatial periodicity. Extensive experiments on the Earth Reanalysis 5 (ERA5) reanalysis dataset demonstrate our model yields a significant improvement over the advanced data-driven models, including PanguWeather and GraphCast, as well as skillful ECMWF systems. Additionally, we empirically show the effectiveness of our model designs and high-quality prediction over spatial and temporal dimensions.

Problem

• Improves Subseasonal-to-Seasonal climate forecasting
• Addresses geometric inductive biases in weather data
• Enhances spatial and temporal prediction accuracy

Innovation

• Geometry-inspired Circular Transformer
• Decomposing weather data by latitude
• Fourier transform in self-attention