🤖 AI Summary
To address the slow convergence and low accuracy of Physics-Informed Neural Networks (PINNs) and Deep Operator Networks (DeepONets) in solving partial differential equations (PDEs)—caused by highly heterogeneous residual decay rates across training points—this work identifies imbalanced residual decay as the primary cause of training failure. We propose a point-wise residual decay rate balancing mechanism with adaptive weighting. Our method introduces a bounded, hyperparameter-free, and low-uncertainty residual equalization criterion, integrating dynamic residual monitoring, gradient-aware weight updates, and a unified PINN/DeepONet co-optimization framework. Evaluated on diverse PDE benchmarks, our approach achieves significantly higher accuracy and faster convergence compared to state-of-the-art adaptive methods, reduces computational cost by over 30%, and cuts training uncertainty by 50%.
📝 Abstract
Physics-informed deep learning has emerged as a promising alternative for solving partial differential equations. However, for complex problems, training these networks can still be challenging, often resulting in unsatisfactory accuracy and efficiency. In this work, we demonstrate that the failure of plain physics-informed neural networks arises from the significant discrepancy in the convergence speed of residuals at different training points, where the slowest convergence speed dominates the overall solution convergence. Based on these observations, we propose a point-wise adaptive weighting method that balances the residual decay rate across different training points. The performance of our proposed adaptive weighting method is compared with current state-of-the-art adaptive weighting methods on benchmark problems for both physics-informed neural networks and physics-informed deep operator networks. Through extensive numerical results we demonstrate that our proposed approach of balanced residual decay rates offers several advantages, including bounded weights, high prediction accuracy, fast convergence speed, low training uncertainty, low computational cost and ease of hyperparameter tuning.