MeTok: An Efficient Meteorological Tokenization with Hyper-Aligned Group Learning for Precipitation Nowcasting

2026-03-14

AI Summary

This study addresses the limitations of conventional location-based meteorological tokenization methods, which neglect the synergistic relationships among meteorological variables and struggle to model complex phenomena such as extreme precipitation. To overcome this, the authors propose MeTok, a distribution-centric meteorological tokenization framework that clusters similar meteorological feature spaces and integrates grouped attention (GA), neighborhood feed-forward networks (N-FFN), and a HyAGTransformer architecture to enable self-aligned feature learning and contextual fusion across diverse precipitation patterns. Experiments on the ERA5 dataset for 6-hour nowcasting of precipitation demonstrate that MeTok improves the Intersection-over-Union (IoU) metric for extreme precipitation prediction by at least 8.2% over existing methods, while also exhibiting strong scalability and stability.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Recently, Transformer-based architectures have advanced meteorological prediction. However, this position-centric tokenizer conflicts with the core principle of meteorological systems, where the weather phenomena undoubtedly involve synergistic interactions among multiple elements while positional information constitutes merely a component of the boundary conditions. This paper focuses primarily on the task of precipitation nowcasting and develops an efficient distribution-centric Meteorological Tokenization (MeTok) scheme, which spatially sequences to group similar meteorological features. Based on the rearrangement, realigned group learning enhances robustness across precipitation patterns, especially extreme ones. Specifically, we introduce the Hyper-Aligned Grouping Transformer (HyAGTransformer) with two key improvements: 1) The Grouping Attention (GA) mechanism uses MeTok to enable self-aligned learning of features from different precipitation patterns; 2) The Neighborhood Feed-Forward Network (N-FFN) integrates adjacent group features, aggregating contextual information to boost patch embedding discriminability. Experiments on the ERA5 dataset for 6-hour forecasts show our method improves the IoU metric by at least 8.2% in extreme precipitation prediction compared to other methods. Additionally, it gains performance with more training data and increased parameters, demonstrating scalability, stability, and superiority over traditional methods.

Problem

• precipitation nowcasting
• meteorological tokenization
• extreme precipitation
• Transformer-based forecasting
• spatiotemporal modeling

Innovation

• Meteorological Tokenization
• Grouping Attention
• Hyper-Aligned Grouping Transformer
• Precipitation Nowcasting
• Distribution-Centric Representation