Improved Physics-informed neural networks loss function regularization with a variance-based term

2024-12-18

AI Summary

Standard mean-squared error (MSE) loss in physics-informed neural networks (PINNs) inadequately suppresses local outliers—e.g., gradient spikes and discontinuities—leading to globally non-uniform solutions and reduced robustness. To address this, we propose a novel variance-sensitive composite loss function that explicitly incorporates the standard deviation of prediction errors into the PINN loss term for the first time. This enables adaptive suppression of high-error regions without introducing auxiliary networks or labeled data. Our method is embedded within a unified framework leveraging physical constraints, automatic differentiation, and PDE solvers for canonical equations: Poisson, Burgers, linear elasticity, and Navier–Stokes. Evaluated across four benchmark problems, the proposed loss reduces maximum error by up to 37% compared to standard MSE, significantly improving global accuracy and solution stability. Crucially, computational overhead remains negligible.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

In machine learning and statistical modeling, the mean square or absolute error is commonly used as an error metric, also called a"loss function."While effective in reducing the average error, this approach may fail to address localized outliers, leading to significant inaccuracies in regions with sharp gradients or discontinuities. This issue is particularly evident in physics-informed neural networks (PINNs), where such localized errors are expected and affect the overall solution. To overcome this limitation, we propose a novel loss function that combines the mean and the standard deviation of the chosen error metric. By minimizing this combined loss function, the method ensures a more uniform error distribution and reduces the impact of localized high-error regions. The proposed loss function is easy to implement and tested on problems of varying complexity: the 1D Poisson equation, the unsteady Burgers' equation, 2D linear elastic solid mechanics, and 2D steady Navier-Stokes equations. Results demonstrate improved solution quality and lower maximum error compared to the standard mean-based loss, with minimal impact on computational time.

Problem

• Addresses localized outliers in PINNs
• Proposes variance-based loss function
• Improves error distribution uniformity

Innovation

• Combines mean and standard deviation
• Ensures uniform error distribution
• Reduces localized high-error regions