Looped SSMs: Depth-Recurrence and Input Reshaping for Time Series Classification

2026-05-15

AI Summary

This work explores two underexplored design dimensions of state space models (SSMs) for time series classification: deep recurrence and input reshaping. By recurrently applying the same SSM module across depth (i.e., deep recurrence) and integrating temporal concatenation or feature-time rechunking strategies at the input stage, the proposed approach substantially enhances model performance. The study introduces deep recurrence into SSMs for the first time, revealing its role as an effective inductive bias, and systematically demonstrates the consistent benefits of input reshaping across both low- and high-dimensional time series. Evaluated on six benchmarks, the method matches or surpasses existing large models with fewer parameters, achieving accuracy gains of 1–6%. The combined effect of both techniques is additive and consistently validated across multiple random seeds.

Abstract

State Space Models (SSMs) are inherently recurrent along the sequence dimension, yet depth-recurrence - reusing the same block repeatedly across layers, as recently applied in looped transformers - has not been explored in this model family. We show that a looped SSM with $k$ parameters iterated $L$ times consistently closely matches or outperforms a standard SSM with $k \cdot L$ independent parameters across four architectures (LRU, S5, LinOSS, LrcSSM) and six time series classification benchmarks, despite operating within a strictly smaller hypothesis space, as we formally establish. Since the larger model contains the looped model as a special case, this dominance cannot be explained by expressivity and instead points to parameter sharing across depth as a beneficial inductive bias that simplifies optimization. These results demonstrate that depth-recurrence is orthogonal to sequence-recurrence and independently beneficial. We further show that input reshaping is an equally neglected design axis: concatenating timesteps for low-dimensional inputs, or flattening and rechunking the joint feature-time dimension for high-dimensional ones, yields accuracy gains of 1-6% across all models, confirmed over 5 random seeds. Both techniques provide standalone improvements that compound when combined, suggesting that depth and input reshaping are two independent and underexplored design axes for SSMs on time series.

Problem

• State Space Models
• depth-recurrence
• input reshaping
• time series classification
• parameter sharing

Innovation

• depth-recurrence
• input reshaping
• state space models
• parameter sharing
• time series classification