Physics-Informed Echo State Networks for Modeling Controllable Dynamical Systems

2024-09-27

arXiv.org

AI Summary

To address overfitting and poor generalization of Echo State Networks (ESNs) in few-shot modeling of nonlinear dynamical systems under external inputs, this paper proposes a Physics-Informed Echo State Network for Controllable systems (PI-ESN-C). It is the first work to jointly embed physical constraints—expressed as ordinary or differential-algebraic equations—with control signals into the ESN architecture, and introduces an adaptive balancing loss mechanism to jointly optimize data fidelity and physical consistency. The method is validated on three benchmark systems: the Van der Pol oscillator, a quadruple-tank process, and a submersible electric pump system. Under few-shot conditions, PI-ESN-C reduces prediction error by up to 92%, effectively mitigating overfitting. It demonstrates strong robustness to parametric uncertainty and achieves superior closed-loop control performance and extrapolation accuracy in Model Predictive Control (MPC) tasks.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Neuro-Symbolic Learning
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems

Abstract

Echo State Networks (ESNs) are recurrent neural networks usually employed for modeling nonlinear dynamic systems with relatively ease of training. By incorporating physical laws into the training of ESNs, Physics-Informed ESNs (PI-ESNs) were proposed initially to model chaotic dynamic systems without external inputs. They require less data for training since Ordinary Differential Equations (ODEs) of the considered system help to regularize the ESN. In this work, the PI-ESN is extended with external inputs to model controllable nonlinear dynamic systems. Additionally, an existing self-adaptive balancing loss method is employed to balance the contributions of the residual regression term and the physics-informed loss term in the total loss function. The experiments with two nonlinear systems modeled by ODEs, the Van der Pol oscillator and the four-tank system, and with one differential-algebraic (DAE) system, an electric submersible pump, revealed that the proposed PI-ESN outperforms the conventional ESN, especially in scenarios with limited data availability, showing that PI-ESNs can regularize an ESN model with external inputs previously trained on just a few datapoints, reducing its overfitting and improving its generalization error (up to 92% relative reduction in the test error). Further experiments demonstrated that the proposed PI-ESN is robust to parametric uncertainties in the ODE equations and that model predictive control using PI-ESN outperforms the one using plain ESN, particularly when training data is scarce.

Problem

• Model controllable nonlinear dynamic systems
• Reduce overfitting with limited data
• Enhance robustness to parametric uncertainties

Innovation

• Physics-Informed ESNs
• Self-adaptive balancing loss
• External inputs integration