RAMS: Residual-based adversarial-gradient moving sample method for scientific machine learning in solving partial differential equations

2025-09-01

AI Summary

In scientific machine learning, solving high-dimensional partial differential equations (PDEs) and learning operators suffer from low sampling efficiency, high computational cost, and insufficient coverage of high-residual regions. To address these challenges, we propose Residual-guided Gradient-based Moving Sampling (RGMS), the first adaptive sampling method integrated into neural operator frameworks. RGMS leverages physics-informed neural networks to estimate residuals in real time and dynamically relocates training samples along the negative gradient direction of the residual, thereby concentrating optimization effort on high-error regions. It is compatible with mainstream sampling strategies and requires no additional labels or predefined distributions. Experiments demonstrate that RGMS significantly accelerates convergence and improves prediction accuracy for both high-dimensional PDE solving and data-driven or physics-constrained operator learning tasks. At equivalent accuracy, it reduces training cost by 30–50%.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Physics-informed neural networks (PINNs) and neural operators, two leading scientific machine learning (SciML) paradigms, have emerged as powerful tools for solving partial differential equations (PDEs). Although increasing the training sample size generally enhances network performance, it also increases computational costs for physics-informed or data-driven training. To address this trade-off, different sampling strategies have been developed to sample more points in regions with high PDE residuals. However, existing sampling methods are computationally demanding for high-dimensional problems, such as high-dimensional PDEs or operator learning tasks. Here, we propose a residual-based adversarial-gradient moving sample (RAMS) method, which moves samples according to the adversarial gradient direction to maximize the PDE residual via gradient-based optimization. RAMS can be easily integrated into existing sampling methods. Extensive experiments, ranging from PINN applied to high-dimensional PDEs to physics-informed and data-driven operator learning problems, have been conducted to demonstrate the effectiveness of RAMS. Notably, RAMS represents the first efficient adaptive sampling approach for operator learning, marking a significant advancement in the SciML field.

Problem

• Improving sampling efficiency in scientific machine learning
• Reducing computational costs for high-dimensional PDE problems
• Enhancing adaptive sampling methods for operator learning

Innovation

• Moves samples via adversarial gradient optimization
• Maximizes PDE residual through gradient-based methods
• Integrates easily with existing adaptive sampling techniques