Unsupervised learning of multiscale switching dynamical system models from multimodal neural data

2025-12-14

AI Summary

Modeling multivariate neural data (e.g., spike trains and local field potentials, LFP) is challenging due to the absence of ground-truth state labels and the presence of latent, multiscale, nonstationary switching dynamics. Method: We propose the first unsupervised multiscale switching dynamical system framework, integrating hidden Markov dynamical systems with multiscale state-space modeling. It jointly estimates spike-LFP likelihoods and identifies cross-scale dynamic regimes via variational inference and EM optimization. Contributions/Results: (1) Learns multiscale switching structure without any labeled states; (2) Enables the first unsupervised, coupled switching model for spike-LFP interactions; (3) Reveals mechanistic insights into how multimodal fusion enhances behavioral decoding. Evaluated on synthetic data and two real-world motor tasks, our model significantly outperforms single-scale switching and static multiscale baselines in behavioral decoding accuracy, demonstrating superior efficacy and robustness.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Multimodal Learning
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Neural population activity often exhibits regime-dependent non-stationarity in the form of switching dynamics. Learning accurate switching dynamical system models can reveal how behavior is encoded in neural activity. Existing switching approaches have primarily focused on learning models from a single neural modality, either continuous Gaussian signals or discrete Poisson signals. However, multiple neural modalities are often recorded simultaneously to measure different spatiotemporal scales of brain activity, and all these modalities can encode behavior. Moreover, regime labels are typically unavailable in training data, posing a significant challenge for learning models of regime-dependent switching dynamics. To address these challenges, we develop a novel unsupervised learning algorithm that learns the parameters of switching multiscale dynamical system models using only multiscale neural observations. We demonstrate our method using both simulations and two distinct experimental datasets with multimodal spike-LFP observations during different motor tasks. We find that our switching multiscale dynamical system models more accurately decode behavior than switching single-scale dynamical models, showing the success of multiscale neural fusion. Further, our models outperform stationary multiscale models, illustrating the importance of tracking regime-dependent non-stationarity in multimodal neural data. The developed unsupervised learning framework enables more accurate modeling of complex multiscale neural dynamics by leveraging information in multimodal recordings while incorporating regime switches. This approach holds promise for improving the performance and robustness of brain-computer interfaces over time and for advancing our understanding of the neural basis of behavior.

Problem

• Learns switching dynamical systems from multimodal neural data
• Models regime-dependent non-stationarity without labeled training data
• Fuses multiscale neural observations to decode behavior accurately

Innovation

• Unsupervised learning algorithm for multimodal neural data
• Switching multiscale dynamical system models without regime labels
• Fusion of multiscale neural observations to decode behavior