Architecture-Optimization Co-Design for Physics-Informed Neural Networks Via Attentive Representations and Conflict-Resolved Gradients

2026-01-19

AI Summary

This work proposes ACR-PINN, a novel physics-informed neural network architecture that addresses the limitations of standard PINNs in solving partial differential equations—namely, insufficient expressive capacity and gradient conflicts arising from multiple physical constraints. By integrating an inter-layer dynamic attention mechanism with a conflict-aware gradient update strategy, ACR-PINN enhances both model expressiveness and training stability while preserving the conventional PINN loss formulation. The approach frames PINN training as a multi-task learning problem, leveraging dynamic attention to enrich feature representations and incorporating a gradient conflict resolution mechanism to guide parameter updates. Extensive evaluations on benchmark problems—including Burgers’, Helmholtz, Klein–Gordon equations, and lid-driven cavity flow—demonstrate that ACR-PINN achieves significantly lower L₂ and L∞ relative errors and faster convergence, confirming its effectiveness and robustness.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Physics-Informed Neural Networks (PINNs) provide a learning-based framework for solving partial differential equations (PDEs) by embedding governing physical laws into neural network training. In practice, however, their performance is often hindered by limited representational capacity and optimization difficulties caused by competing physical constraints and conflicting gradients. In this work, we study PINN training from a unified architecture-optimization perspective. We first propose a layer-wise dynamic attention mechanism to enhance representational flexibility, resulting in the Layer-wise Dynamic Attention PINN (LDA-PINN). We then reformulate PINN training as a multi-task learning problem and introduce a conflict-resolved gradient update strategy to alleviate gradient interference, leading to the Gradient-Conflict-Resolved PINN (GC-PINN). By integrating these two components, we develop the Architecture-Conflict-Resolved PINN (ACR-PINN), which combines attentive representations with conflict-aware optimization while preserving the standard PINN loss formulation. Extensive experiments on benchmark PDEs, including the Burgers, Helmholtz, Klein-Gordon, and lid-driven cavity flow problems, demonstrate that ACR-PINN achieves faster convergence and significantly lower relative $L_2$ and $L_\infty$ errors than standard PINNs. These results highlight the effectiveness of architecture-optimization co-design for improving the robustness and accuracy of PINN-based solvers.

Problem

• Physics-Informed Neural Networks
• Partial Differential Equations
• Gradient Conflict
• Representational Capacity
• Optimization Difficulty

Innovation

• Physics-Informed Neural Networks
• Attention Mechanism
• Gradient Conflict Resolution
• Architecture-Optimization Co-Design
• Multi-Task Learning