Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm

2026-02-09

AI Summary

This work proposes an efficient shallow physics-informed neural network (PINN) framework—employing only two hidden layers—to address the high computational cost and slow convergence of conventional deep PINNs in solving forward and inverse problems of nonlinear partial differential equations. By modeling the network as a nonlinear system and leveraging the Levenberg-Marquardt (LM) second-order optimization algorithm, the method constructs an exact Jacobian matrix using analytical derivatives of the input variables. Evaluated on benchmark problems including Burgers’, Schrödinger, Allen–Cahn, and three-dimensional Bratu equations, the approach consistently outperforms BFGS in terms of convergence speed, solution accuracy, and final loss, thereby challenging the prevailing notion that PINNs must be deep to be effective.

Technology Category

• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This work investigates the use of shallow physics-informed neural networks (PINNs) for solving forward and inverse problems of nonlinear partial differential equations (PDEs). By reformulating PINNs as nonlinear systems, the Levenberg-Marquardt (LM) algorithm is employed to efficiently optimize the network parameters. Analytical expressions for the neural network derivatives with respect to the input variables are derived, enabling accurate and efficient computation of the Jacobian matrix required by LM. The proposed approach is tested on several benchmark problems, including the Burgers, Schr\"odinger, Allen-Cahn, and three-dimensional Bratu equations. Numerical results demonstrate that LM significantly outperforms BFGS in terms of convergence speed, accuracy, and final loss values, even when using shallow network architectures with only two hidden layers. These findings indicate that, for a wide class of PDEs, shallow PINNs combined with efficient second-order optimization methods can provide accurate and computationally efficient solutions for both forward and inverse problems.

Problem

• physics-informed neural networks
• shallow networks
• nonlinear partial differential equations
• forward problems
• inverse problems

Innovation

• physics-informed neural networks
• Levenberg-Marquardt algorithm
• shallow neural networks
• nonlinear PDEs
• second-order optimization