HyResPINNs: Adaptive Hybrid Residual Networks for Learning Optimal Combinations of Neural and RBF Components for Physics-Informed Modeling

2024-10-04

arXiv.org

AI Summary

Physical-informed neural networks (PINNs) suffer from sensitivity to training point distribution and network architecture, limiting their accuracy in solving partial differential equations (PDEs). To address this, we propose HyResPINNs—a novel adaptive hybrid residual network. HyResPINNs introduces learnable hybrid residual blocks that jointly integrate deep neural networks and radial basis function (RBF) networks, dynamically optimizing both the inter-network weights and cross-block connections. Training is performed end-to-end using a PDE-driven physics-constrained loss. Experiments on the Allen–Cahn and Darcy flow equations demonstrate that HyResPINNs achieves improvements of several orders of magnitude in solution accuracy over standard PINNs and related methods, while significantly enhancing robustness. Crucially, this performance gain incurs only a marginal increase in training cost. HyResPINNs thus effectively bridges the accuracy and reliability gap between classical numerical solvers and AI-driven PDE solvers.

Technology Category

• Machine Learning: Hardware-aware ML
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Physics-informed neural networks (PINNs) are an increasingly popular class of techniques for the numerical solution of partial differential equations (PDEs), where neural networks are trained using loss functions regularized by relevant PDE terms to enforce physical constraints. We present a new class of PINNs called HyResPINNs, which augment traditional PINNs with adaptive hybrid residual blocks that combine the outputs of a standard neural network and a radial basis function (RBF) network. A key feature of our method is the inclusion of adaptive combination parameters within each residual block, which dynamically learn to weigh the contributions of the neural network and RBF network outputs. Additionally, adaptive connections between residual blocks allow for flexible information flow throughout the network. We show that HyResPINNs are more robust to training point locations and neural network architectures than traditional PINNs. Moreover, HyResPINNs offer orders of magnitude greater accuracy than competing methods on certain problems, with only modest increases in training costs. We demonstrate the strengths of our approach on challenging PDEs, including the Allen-Cahn equation and the Darcy-Flow equation. Our results suggest that HyResPINNs effectively bridge the gap between traditional numerical methods and modern machine learning-based solvers.

Problem

• HyResPINNs integrate neural and RBF networks for PDEs.
• Adaptive hybrid residual blocks dynamically weight network contributions.
• HyResPINNs outperform baseline methods in solving diverse PDEs.

Innovation

• Hybrid neural and RBF networks
• Adaptive residual block parameters
• Superior PDE solving accuracy