Extreme Adaptive Transformer for Time Series Forecasting

2026-07-02

AI Summary

This work addresses the challenge of inadequately modeling rare yet high-impact extreme events in time series forecasting, particularly in hydrological streamflow prediction characterized by highly skewed distributions. To this end, the authors propose Exformer, a novel framework that explicitly captures dependencies between normal and extreme events within a Transformer architecture. Central to Exformer is an extreme-adaptive attention mechanism comprising three sparse attention components—Local, Stride, and Extreme—which respectively model short-term dynamics, periodic patterns, and extreme-event-related dependencies. Experimental results on four real-world hydrological datasets demonstrate that Exformer significantly outperforms state-of-the-art models in 3-day-ahead forecasting tasks, effectively enhancing predictive accuracy for extreme events in imbalanced time series.

Abstract

Time series forecasting remains challenging when the underlying data contain rare but critical extreme events. This issue is particularly important in hydrologic forecasting, where streamflow distributions are often highly skewed and extreme peaks can have substantial impacts on flood monitoring, water resource management, and early warning systems. Although Transformer-based forecasting models have achieved strong performance by modeling long-range temporal dependencies, they typically treat all time points uniformly and may therefore underrepresent rare extreme patterns. In this paper, we propose the Extreme-Adaptive Transformer (Exformer), a forecasting framework designed to explicitly model temporal dependencies involving both normal and extreme events. Exformer introduces an extreme-adaptive attention mechanism composed of three sparse components: Local, Stride, and Extreme. The Local and Stride components capture short-term and periodic temporal dependencies, respectively, while the Extreme component selectively models event-aware dependencies between normal and extreme streamflow patterns. Experiments on four real-world hydrologic streamflow datasets show that Exformer achieves superior 3-day forecasting performance compared with state-of-the-art baselines. Our findings demonstrate that explicitly incorporating extreme-aware attention improves the forecasting capacity of Transformer models on imbalanced time series with rare but consequential events.

Problem

• time series forecasting
• extreme events
• hydrologic forecasting
• imbalanced time series
• Transformer

Innovation

• Extreme-Adaptive Attention
• Sparse Attention Mechanism
• Time Series Forecasting
• Extreme Events Modeling
• Transformer Architecture