Mitigating Gradient Pathology in PINNs through Aligned Constraint

2026-05-24

AI Summary

This work addresses the persistent challenge in physics-informed neural networks (PINNs) where conflicting gradients between PDE residuals and boundary constraints often trap optimization in poor local minima. The study systematically uncovers, for the first time, the underlying causes of this gradient pathology and introduces the Constraint Alignment and Manifold Lifting (CAML) framework. CAML mitigates gradient conflicts by reformulating zeroth-order terms into geometrically agnostic alignment constraints and incorporates a delay factor to steer optimizers away from high-curvature regions of the loss landscape. By transcending the limitations of existing adaptive weighting and hard-constraint strategies, the proposed method substantially enhances numerical stability and solution efficiency across complex PINN problems. The implementation is publicly released to facilitate reproducibility and further research.

Abstract

While Physics-Informed Neural Networks (PINNs) are powerful for solving Partial Differential Equations (PDEs), their training is often paralyzed by gradient pathology. The gradients from the PDE residuals and boundary constraints oppose each other, trapping the model in local minima. Current solutions, such as adaptive weighting or hard constraints, either fail to fundamentally resolve this ill-conditioning or are limited to simple geometries. In this study, we systematically analyze the possible causes of this gradient pathology from the perspectives of loss landscapes and optimization dynamics. Based on the obtained conclusion, we propose Constraint-Aligned loss with Manifold Lifting (CAML). By reformulating all zeroth-order terms into aligned constraints, our method effectively mitigates gradient conflicts. In addition, we introduce a delay factor to help the optimizer skip the high-curvature area. Experiments demonstrate that our CAML significantly enhances numerical stability and efficiency in highly complex PINN problems. Our code is open-sourced on https://github.com/YichenLuo-0/CAML.

Problem

• gradient pathology
• Physics-Informed Neural Networks
• PDEs
• optimization dynamics
• loss landscape

Innovation

• Physics-Informed Neural Networks
• Gradient Pathology
• Constraint Alignment
• Manifold Lifting
• Optimization Dynamics