Neural filtering for Neural Network-based Models of Dynamic Systems

2024-09-20

arXiv.org

AI Summary

Neural networks often suffer from error accumulation in long-term prediction of dynamical systems, leading to divergent state estimates. To address this, we propose a differentiable neural filter that, for the first time, embeds the extended Kalman filter (EKF) principle into a neural modeling framework, enabling end-to-end joint optimization of data-driven prediction and physics-informed observation. The filter integrates a neural forward model with an observation-update mechanism, supporting gradient-based training while preserving physical consistency and adaptively correcting prediction bias. Experiments on four nonlinear dynamical systems demonstrate that our method achieves significantly higher long-term prediction accuracy than purely neural baselines. Moreover, it effectively constrains the estimated state covariance, enhancing robustness and reliability under uncertainty. This work bridges classical filtering theory and deep learning, offering a principled approach to physically grounded sequential estimation.

Technology Category

• Intelligent Robots: State Estimation
• Reasoning under Uncertainty: Stochastic Optimization
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

The application of neural networks in modeling dynamic systems has become prominent due to their ability to estimate complex nonlinear functions. Despite their effectiveness, neural networks face challenges in long-term predictions, where the prediction error diverges over time, thus degrading their accuracy. This paper presents a neural filter to enhance the accuracy of long-term state predictions of neural network-based models of dynamic systems. Motivated by the extended Kalman filter, the neural filter combines the neural network state predictions with the measurements from the physical system to improve the estimated state's accuracy. The neural filter's improvements in prediction accuracy are demonstrated through applications to four nonlinear dynamical systems. Numerical experiments show that the neural filter significantly improves prediction accuracy and bounds the state estimate covariance, outperforming the neural network predictions.

Problem

• Improves long-term prediction accuracy in dynamic systems
• Combines neural predictions with physical measurements
• Enhances poorly trained models to adequate performance

Innovation

• Neural filter enhances long-term prediction accuracy
• Combines neural predictions with physical measurements
• Improves poorly trained models to adequate levels