Learning and discovering multiple solutions using physics-informed neural networks with random initialization and deep ensemble

2025-03-08

AI Summary

Nonlinear differential equations—such as the Allen–Cahn equation and cavity flow PDEs—often admit multiple stable or unstable solutions under identical boundary/initial conditions, posing a fundamental challenge for conventional numerical methods (e.g., time-marching or Newton-type solvers), which are highly sensitive to initial guesses and thus struggle to systematically uncover all coexisting solutions. Method: This paper proposes a physics-informed neural network (PINN) framework leveraging stochastic weight initialization and deep ensemble strategies. It establishes a novel two-stage paradigm: PINNs generate diverse initial guesses via random initialization, followed by refinement using traditional numerical solvers. Contribution/Results: We first demonstrate that stochastic initialization is decisive for solution diversity in PINNs. The method automatically discovers multiple coexisting solutions across diverse nonlinear ODEs and PDEs, significantly improving robustness, efficiency, and interpretability in multiplicity detection. This work opens a new avenue for leveraging PINNs in exploratory analysis of solution landscapes.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We explore the capability of physics-informed neural networks (PINNs) to discover multiple solutions. Many real-world phenomena governed by nonlinear differential equations (DEs), such as fluid flow, exhibit multiple solutions under the same conditions, yet capturing this solution multiplicity remains a significant challenge. A key difficulty is giving appropriate initial conditions or initial guesses, to which the widely used time-marching schemes and Newton's iteration method are very sensitive in finding solutions for complex computational problems. While machine learning models, particularly PINNs, have shown promise in solving DEs, their ability to capture multiple solutions remains underexplored. In this work, we propose a simple and practical approach using PINNs to learn and discover multiple solutions. We first reveal that PINNs, when combined with random initialization and deep ensemble method -- originally developed for uncertainty quantification -- can effectively uncover multiple solutions to nonlinear ordinary and partial differential equations (ODEs/PDEs). Our approach highlights the critical role of initialization in shaping solution diversity, addressing an often-overlooked aspect of machine learning for scientific computing. Furthermore, we propose utilizing PINN-generated solutions as initial conditions or initial guesses for conventional numerical solvers to enhance accuracy and efficiency in capturing multiple solutions. Extensive numerical experiments, including the Allen-Cahn equation and cavity flow, where our approach successfully identifies both stable and unstable solutions, validate the effectiveness of our method. These findings establish a general and efficient framework for addressing solution multiplicity in nonlinear differential equations.

Problem

• Discover multiple solutions in nonlinear differential equations.
• Address sensitivity to initial conditions in numerical methods.
• Enhance accuracy and efficiency using PINN-generated initial guesses.

Innovation

• Physics-informed neural networks with random initialization
• Deep ensemble method for multiple solution discovery
• PINN-generated solutions as initial conditions for numerical solvers