An Imbalanced Learning-based Sampling Method for Physics-informed Neural Networks

2025-01-20

AI Summary

To address the accuracy degradation and excessive memory consumption caused by highly imbalanced residual distributions in Physics-Informed Neural Networks (PINNs), this paper proposes RSmote—the first locally adaptive oversampling method for PINN residual sampling inspired by SMOTE. RSmote constructs residual-driven neighborhoods and performs interpolation-based sample generation, supported by theoretical convergence analysis, thereby significantly reducing memory footprint and enhancing training stability. Experiments on diverse high-dimensional PDE benchmarks demonstrate that RSmote achieves accuracy comparable to or exceeding state-of-the-art methods (e.g., RAD), while drastically lowering memory usage—making it especially suitable for resource-constrained complex physics modeling. The core contribution lies in establishing, for the first time, a theoretical linkage between imbalanced learning and residual sampling in PINNs, and achieving synergistic optimization of computational efficiency and generalization performance.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

This paper introduces Residual-based Smote (RSmote), an innovative local adaptive sampling technique tailored to improve the performance of Physics-Informed Neural Networks (PINNs) through imbalanced learning strategies. Traditional residual-based adaptive sampling methods, while effective in enhancing PINN accuracy, often struggle with efficiency and high memory consumption, particularly in high-dimensional problems. RSmote addresses these challenges by targeting regions with high residuals and employing oversampling techniques from imbalanced learning to refine the sampling process. Our approach is underpinned by a rigorous theoretical analysis that supports the effectiveness of RSmote in managing computational resources more efficiently. Through extensive evaluations, we benchmark RSmote against the state-of-the-art Residual-based Adaptive Distribution (RAD) method across a variety of dimensions and differential equations. The results demonstrate that RSmote not only achieves or exceeds the accuracy of RAD but also significantly reduces memory usage, making it particularly advantageous in high-dimensional scenarios. These contributions position RSmote as a robust and resource-efficient solution for solving complex partial differential equations, especially when computational constraints are a critical consideration.

Problem

• Imbalanced Data
• Physical Knowledge Neural Network
• Memory Consumption

Innovation

• RSmote
• Imbalanced Learning
• Physics-Informed Neural Networks