Efficient Transformed Gaussian Process State-Space Models for Non-Stationary High-Dimensional Dynamical Systems

2025-03-24

AI Summary

To address the low computational efficiency, poor scalability, and overly restrictive stationarity assumptions of Gaussian Process State-Space Models (GPSSMs) in modeling high-dimensional nonstationary dynamical systems, this paper proposes the Efficient Transformation GPSSM (ETGPSSM). Methodologically, it introduces— for the first time—the coupling of normalizing flows with shared Gaussian processes to construct a flexible, nonstationary latent dynamics prior; develops a generative-process-based variational inference framework; and integrates the Ensemble Kalman Filter (EnKF) to circumvent the absence of closed-form solutions in latent-transformed GPs. Furthermore, Bayesian neural networks are incorporated to enhance representational capacity. Experiments on synthetic and real-world benchmarks demonstrate that ETGPSSM significantly outperforms existing GPSSMs and neural baselines in both high-dimensional state estimation and time-series forecasting, achieving superior accuracy while maintaining high computational efficiency.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Time-Series/Data Streams
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Gaussian process state-space models (GPSSMs) have emerged as a powerful framework for modeling dynamical systems, offering interpretable uncertainty quantification and inherent regularization. However, existing GPSSMs face significant challenges in handling high-dimensional, non-stationary systems due to computational inefficiencies, limited scalability, and restrictive stationarity assumptions. In this paper, we propose an efficient transformed Gaussian process state-space model (ETGPSSM) to address these limitations. Our approach leverages a single shared Gaussian process (GP) combined with normalizing flows and Bayesian neural networks, enabling efficient modeling of complex, high-dimensional state transitions while preserving scalability. To address the lack of closed-form expressions for the implicit process in the transformed GP, we follow its generative process and introduce an efficient variational inference algorithm, aided by the ensemble Kalman filter (EnKF), to enable computationally tractable learning and inference. Extensive empirical evaluations on synthetic and real-world datasets demonstrate the superior performance of our ETGPSSM in system dynamics learning, high-dimensional state estimation, and time-series forecasting, outperforming existing GPSSMs and neural network-based methods in both accuracy and computational efficiency.

Problem

• Modeling high-dimensional non-stationary dynamical systems efficiently
• Overcoming computational inefficiencies in Gaussian process state-space models
• Enabling scalable inference for complex state transitions

Innovation

• Combines shared GP with normalizing flows
• Uses efficient variational inference algorithm
• Integrates ensemble Kalman filter for learning