Spectral Bias Mitigation via xLSTM-PINN: Memory-Gated Representation Refinement for Physics-Informed Learning

2025-11-16

AI Summary

This work addresses three persistent challenges in physics-informed neural networks (PINNs) for PDE solving: spectral bias, residual-data imbalance, and poor extrapolation capability. To this end, we propose xLSTM-PINN—a novel framework integrating physical constraints via end-to-end training. Its core innovations are: (1) a gated cross-scale xLSTM memory architecture enabling multi-scale feature extraction and dynamic frequency-domain reshaping; (2) a staged frequency-curriculum learning strategy to broaden the resolvable spectral bandwidth; and (3) an adaptive residual reweighting mechanism to mitigate loss-term imbalance. Evaluated on four benchmark PDE tasks, xLSTM-PINN significantly reduces MSE, RMSE, MAE, and MaxAE; accelerates convergence for high-wavenumber components; enhances boundary transition sharpness and high-frequency structural fidelity; and improves both extrapolation robustness and training stability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Physics-informed learning for PDEs is surging across scientific computing and industrial simulation, yet prevailing methods face spectral bias, residual-data imbalance, and weak extrapolation. We introduce a representation-level spectral remodeling xLSTM-PINN that combines gated-memory multiscale feature extraction with adaptive residual-data weighting to curb spectral bias and strengthen extrapolation. Across four benchmarks, we integrate gated cross-scale memory, a staged frequency curriculum, and adaptive residual reweighting, and verify with analytic references and extrapolation tests, achieving markedly lower spectral error and RMSE and a broader stable learning-rate window. Frequency-domain benchmarks show raised high-frequency kernel weights and a right-shifted resolvable bandwidth, shorter high-k error decay and time-to-threshold, and narrower error bands with lower MSE, RMSE, MAE, and MaxAE. Compared with the baseline PINN, we reduce MSE, RMSE, MAE, and MaxAE across all four benchmarks and deliver cleaner boundary transitions with attenuated high-frequency ripples in both frequency and field maps. This work suppresses spectral bias, widens the resolvable band and shortens the high-k time-to-threshold under the same budget, and without altering AD or physics losses improves accuracy, reproducibility, and transferability.

Problem

• Mitigating spectral bias in physics-informed neural networks
• Improving extrapolation capabilities for PDE solutions
• Balancing residual and data loss in scientific simulations

Innovation

• Gated memory multiscale feature extraction for spectral remodeling
• Adaptive residual-data weighting to balance learning objectives
• Staged frequency curriculum to enhance high-frequency resolution