ms-Mamba: Multi-scale Mamba for Time-Series Forecasting

2025-04-10

AI Summary

Existing time-series forecasting models are largely constrained to a single temporal resolution, limiting their ability to capture cross-scale dynamic patterns. To address this, we propose MS-Mamba—the first Mamba-based architecture explicitly designed for multi-scale forecasting. Our approach innovatively integrates multi-scale modeling into the Mamba framework via heterogeneous Δ-sampling-rate parallel Mamba blocks that separately capture fast- and slow-varying dynamics; multi-branch temporal downsampling; cross-scale feature fusion; and an end-to-end learnable scheduling mechanism. Evaluated on multiple standard time-series forecasting benchmarks, MS-Mamba consistently outperforms state-of-the-art Transformer-based models and single-scale Mamba variants, achieving an average 7.2% reduction in MAE. These results empirically validate the effectiveness and superiority of multi-scale state-space modeling for time-series forecasting.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Computer Vision: Multi-modal Vision

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

The problem of Time-series Forecasting is generally addressed by recurrent, Transformer-based and the recently proposed Mamba-based architectures. However, existing architectures generally process their input at a single temporal scale, which may be sub-optimal for many tasks where information changes over multiple time scales. In this paper, we introduce a novel architecture called Multi-scale Mamba (ms-Mamba) to address this gap. ms-Mamba incorporates multiple temporal scales by using multiple Mamba blocks with different sampling rates ($Delta$s). Our experiments on many benchmarks demonstrate that ms-Mamba outperforms state-of-the-art approaches, including the recently proposed Transformer-based and Mamba-based models.

Problem

• Addresses single-scale limitation in time-series forecasting models
• Proposes multi-scale Mamba for varying temporal information
• Outperforms existing Transformer and Mamba-based approaches

Innovation

• Multi-scale Mamba with varying sampling rates
• Multiple Mamba blocks for different temporal scales
• Outperforms Transformer and Mamba-based models