TIDES: Implicit Time-Awareness in Selective State Space Models

2026-05-10

AI Summary

This work addresses a critical limitation in existing selective state space models—such as Mamba—which lose the physical interpretability of time steps by treating them as arbitrary input-dependent functions, thereby struggling with irregularly sampled time series. While continuous-time models like S5 preserve temporal semantics, they are constrained by linear time-invariant dynamics and lack per-token expressivity. To overcome these issues, we propose TIDES, the first selective state space model that decouples input dependence from time steps and instead introduces an input-dependent diagonal state matrix. This design retains the physical meaning of time steps while achieving high representational capacity. TIDES natively supports irregular sampling, attains the best average rank on UEA classification and Physiome-ODE regression benchmarks, and demonstrates strong out-of-distribution extrapolation on a newly introduced Fading Flash task involving unseen time intervals.

Abstract

Selective state space models (SSMs), such as Mamba, achieve strong per-token expressivity by making the time discretization step $\TildeΔ$ a learned function of the input. However, in doing so, $\TildeΔ$ ceases to represent a physical sampling interval, limiting its irregular time series modeling capability. Continuous-time SSMs, such as S5, preserve the physical meaning of $\TildeΔ$ and handle irregular timestamps natively ($\TildeΔ\equivΔ)$, but their dynamics remain linear time-invariant (LTI), limiting per-token expressivity. We propose \textbf{TIDES}, a selective SSM variant that reconciles selective and continuous architectures by moving input-dependence off the step size and onto the diagonal state matrix. As a result, $\TildeΔ$ retains its physical meaning, tied to the state discretization, allowing the model to handle irregular timestamps natively without sacrificing the per-token expressivity that makes selective SSMs effective. We show this on a novel \emph{Fading Flash} experimental benchmark, a compact controlled diagnostic for sequence models that jointly tests input-dependence and extrapolation to out-of-distribution $Δ$ values, and isolates the distinct failure modes of current state-of-the-art architectures that TIDES avoids by construction. On large-scale benchmarks, TIDES sets the new state-of-the-art average rank on UEA time-series classification and the Physiome-ODE regression benchmark. Code available at: https://github.com/TaylanSoydan/TIDES.

Problem

• selective state space models
• irregular time series
• time-awareness
• per-token expressivity
• continuous-time modeling

Innovation

• Selective State Space Models
• Irregular Time Series
• Continuous-time Modeling
• Input-dependent Dynamics
• Time-aware Deep Learning