WaveSSM: Multiscale State-Space Models for Non-stationary Signal Attention

2026-02-25

AI Summary

Existing state space models based on polynomial bases struggle to effectively capture nonstationary signals with localized or transient structures. This work proposes a multiscale state space model grounded in wavelet frames, which for the first time incorporates the local support property of wavelets into state space modeling, replacing conventional global polynomial bases. This substitution substantially enhances the model’s capacity to represent localized temporal dynamics. By integrating the continuous-time projection framework of HiPPO, the approach achieves both efficiency and stability in long-sequence modeling. Empirical evaluations on real-world datasets featuring transient structures—such as PTB-XL electrocardiograms and raw audio from Speech Commands—demonstrate superior performance over orthogonal-basis methods like S4.

Technology Category

• Machine Learning: Multimodal Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Large-scale security measurements

Abstract

State-space models (SSMs) have emerged as a powerful foundation for long-range sequence modeling, with the HiPPO framework showing that continuous-time projection operators can be used to derive stable, memory-efficient dynamical systems that encode the past history of the input signal. However, existing projection-based SSMs often rely on polynomial bases with global temporal support, whose inductive biases are poorly matched to signals exhibiting localized or transient structure. In this work, we introduce \emph{WaveSSM}, a collection of SSMs constructed over wavelet frames. Our key observation is that wavelet frames yield a localized support on the temporal dimension, useful for tasks requiring precise localization. Empirically, we show that on equal conditions, \textit{WaveSSM} outperforms orthogonal counterparts as S4 on real-world datasets with transient dynamics, including physiological signals on the PTB-XL dataset and raw audio on Speech Commands.

Problem

• state-space models
• non-stationary signals
• wavelet frames
• transient dynamics
• temporal localization

Innovation

• Wavelet frames
• State-space models
• Non-stationary signals
• Temporal localization
• Transient dynamics