Stability Analysis of Physics-Informed Neural Networks for Stiff Linear Differential Equations

2024-08-27

arXiv.org

AI Summary

Physics-informed neural networks (PINNs) suffer from poor linear stability when solving stiff initial-value problems of ordinary differential equations (ODEs). Method: We propose physics-informed random projection neural networks (PI-RPNNs), introducing a novel random projection framework with both multi-point and single-point collocation strategies. Contribution/Results: We provide the first rigorous proof that multi-point PI-RPNNs achieve asymptotic stability for highly stiff linear ODE systems, while single-point PI-RPNNs satisfy A-stability. This establishes the first unified convergence and stability theory for RP-PINNs applied to stiff linear ODEs and parabolic partial differential equations (PDEs). Compared with classical implicit methods—including backward Euler, Crank–Nicolson, and Radau schemes—PI-RPNNs deliver significantly higher numerical accuracy and computational efficiency over wide step-size ranges. The method overcomes the longstanding trade-off between stability and accuracy in traditional numerical solvers, providing both theoretical foundations and practical tools for reliable PINN-based simulation of stiff problems.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Learning with Manifolds
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We present a stability analysis of Physics-Informed Neural Networks (PINNs) coupled with random projections, for the numerical solution of (stiff) linear differential equations. For our analysis, we consider systems of linear ODEs, and linear parabolic PDEs. We prove that properly designed PINNs offer consistent and asymptotically stable numerical schemes, thus convergent schemes. In particular, we prove that multi-collocation random projection PINNs guarantee asymptotic stability for very high stiffness and that single-collocation PINNs are $A$-stable. To assess the performance of the PINNs in terms of both numerical approximation accuracy and computational cost, we compare it with other implicit schemes and in particular backward Euler, the midpoint, trapezoidal (Crank-Nikolson), the 2-stage Gauss scheme and the 2 and 3 stages Radau schemes. We show that the proposed PINNs outperform the above traditional schemes, in both numerical approximation accuracy and importantly computational cost, for a wide range of step sizes.

Problem

• Analyzing stability of PI-RPNNs for ODE solutions
• Proving PI-RPNNs as uniform approximators for ODEs
• Ensuring consistent and stable numerical schemes via PI-RPNNs

Innovation

• Physics-informed random projection neural networks
• Uniform approximators for ODE solutions
• Multi-collocation ensures asymptotic stability