A Compact Model for Large-Scale Time Series Forecasting

2025-02-28

AI Summary

Large-scale forecasting of high-dimensional spatiotemporal time series—e.g., traffic, financial, and ride-hailing demand data—faces two key challenges: excessive model redundancy and insufficient modeling of intra-cycle fine-grained temporal dependencies. To address these, we propose UltraSTF, the first framework integrating a *cross-cycle forecasting module* with an *ultra-compact shape dictionary*. The former leverages attention-driven cross-cycle shape matching to enhance intra-cycle dynamic dependency learning; the latter represents diverse temporal patterns using minimal parameters, synergistically incorporating sparse attention and channel-independent prediction. Evaluated on the LargeST benchmark, UltraSTF achieves state-of-the-art (SOTA) accuracy while reducing parameter count to just 0.2% of the second-best method—significantly advancing the accuracy-efficiency Pareto frontier for large-scale spatiotemporal forecasting.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Machine Learning: Time-Series/Data Streams
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Spatio-temporal data, which commonly arise in real-world applications such as traffic monitoring, financial transactions, and ride-share demands, represent a special category of multivariate time series. They exhibit two distinct characteristics: high dimensionality and commensurability across spatial locations. These attributes call for computationally efficient modeling approaches and facilitate the use of univariate forecasting models in a channel-independent fashion. SparseTSF, a recently introduced competitive univariate forecasting model, harnesses periodicity to achieve compactness by concentrating on cross-period dynamics, thereby extending the Pareto frontier with respect to model size and predictive performance. Nonetheless, it underperforms on spatio-temporal data due to an inadequate capture of intra-period temporal dependencies. To address this shortcoming, we propose UltraSTF, which integrates a cross-period forecasting module with an ultra-compact shape bank component. Our model effectively detects recurring patterns in time series through the attention mechanism of the shape bank component, thereby strengthening its ability to learn intra-period dynamics. UltraSTF achieves state-of-the-art performance on the LargeST benchmark while employing fewer than 0.2% of the parameters required by the second-best approaches, thus further extending the Pareto frontier of existing methods.

Problem

• Addresses underperformance in spatio-temporal data forecasting.
• Enhances intra-period temporal dependency capture in time series.
• Achieves compactness and state-of-the-art performance with minimal parameters.

Innovation

• Integrates cross-period forecasting with shape bank
• Uses attention mechanism for pattern detection
• Achieves high performance with minimal parameters