Time-Varying Deep State Space Models for Sequences with Switching Dynamics

2026-05-14

AI Summary

This work addresses the challenge that traditional time-invariant models struggle to effectively capture switching dynamics in time-varying systems. To overcome this limitation, the authors propose a neural network–based time-varying state-space model that incorporates a learnable dictionary of time-varying basis functions. This design flexibly represents diverse temporal evolution patterns of system dynamics while maintaining manageable computational complexity, substantially enhancing the model’s capacity to capture switching sequences. The study further reveals an optimal allocation strategy for time-varying degrees of freedom across model components. Experimental results demonstrate that the proposed model consistently outperforms existing time-invariant approaches on both synthetic switching systems and speech denoising tasks, confirming its effectiveness and strong generalization capability.

Abstract

The identification and modeling of time-varying systems is a fundamental challenge in signal processing and system identification. To address this challenge, we propose a class of time-varying state-space model (SSM) based neural networks in which the neurons'states are governed by time-varying dynamics. The proposed model provides the learnable time-varying dynamics through a dictionary of basis functions, where each basis function evolves differently over time. We evaluate the proposed approach on both synthetic data from switching systems and a speech denoising task where real audio is corrupted with switching dynamics noise. The results show that the proposed time-varying model consistently outperforms its time-invariant counterparts while maintaining comparable computational complexity. Our investigations also reveal which aspects of the time-varying dynamics of the data most need to be captured by the proposed time-invariant models, how the additional freedom provided by time-varying basis functions should be allocated across model components, and to what extent larger models can compensate for time-invariant limitations.

Problem

• time-varying systems
• switching dynamics
• state-space models
• system identification
• signal processing

Innovation

• time-varying dynamics
• state-space models
• switching systems
• basis functions
• neural networks