High-Dynamic Radar Sequence Prediction for Weather Nowcasting Using Spatiotemporal Coherent Gaussian Representation

2025-02-17

AI Summary

Accurate nowcasting of highly dynamic 3D radar echo sequences remains challenging, as existing methods predominantly rely on 2D single-layer modeling, failing to simultaneously achieve high accuracy, computational efficiency, and faithful 3D volumetric representation. Method: This paper proposes the first end-to-end 3D volumetric radar sequence forecasting framework. Its core innovations are: (1) Spatio-Temporal Consistent Gaussian Splatting (STC-GS), a geometry-aware representation enabling stable cross-frame tracking of radar voxels and improving spatial resolution by over 16×; and (2) Gaussian-enhanced Mamba (GauMamba), integrating differentiable Gaussian splatting, spatio-temporal consistency constraints, state-space modeling, and Gaussian token memory for efficient 3D encoding and long-range temporal reasoning. Results: Evaluated on complex weather scenarios—including severe convection and squall lines—the framework significantly outperforms state-of-the-art methods in both prediction accuracy and computational efficiency.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Weather nowcasting is an essential task that involves predicting future radar echo sequences based on current observations, offering significant benefits for disaster management, transportation, and urban planning. Current prediction methods are limited by training and storage efficiency, mainly focusing on 2D spatial predictions at specific altitudes. Meanwhile, 3D volumetric predictions at each timestamp remain largely unexplored. To address such a challenge, we introduce a comprehensive framework for 3D radar sequence prediction in weather nowcasting, using the newly proposed SpatioTemporal Coherent Gaussian Splatting (STC-GS) for dynamic radar representation and GauMamba for efficient and accurate forecasting. Specifically, rather than relying on a 4D Gaussian for dynamic scene reconstruction, STC-GS optimizes 3D scenes at each frame by employing a group of Gaussians while effectively capturing their movements across consecutive frames. It ensures consistent tracking of each Gaussian over time, making it particularly effective for prediction tasks. With the temporally correlated Gaussian groups established, we utilize them to train GauMamba, which integrates a memory mechanism into the Mamba framework. This allows the model to learn the temporal evolution of Gaussian groups while efficiently handling a large volume of Gaussian tokens. As a result, it achieves both efficiency and accuracy in forecasting a wide range of dynamic meteorological radar signals. The experimental results demonstrate that our STC-GS can efficiently represent 3D radar sequences with over $16 imes$ higher spatial resolution compared with the existing 3D representation methods, while GauMamba outperforms state-of-the-art methods in forecasting a broad spectrum of high-dynamic weather conditions.

Problem

• Predict 3D radar sequences for weather nowcasting
• Enhance efficiency and accuracy in meteorological forecasting
• Utilize SpatioTemporal Coherent Gaussian Splatting for dynamic representation

Innovation

• SpatioTemporal Coherent Gaussian Splatting
• GauMamba for weather forecasting
• 3D radar sequence prediction