Neuro-Spectral Architectures for Causal Physics-Informed Networks

2025-09-05

AI Summary

Traditional MLP-based physics-informed neural networks (PINNs) often diverge when solving complex initial-value problems, yielding unphysical, causality-violating solutions and exhibiting low-frequency spectral bias. To address these issues, we propose NeuSA—a novel PINN architecture that synergistically integrates spectral methods with neural ordinary differential equations (NODEs). NeuSA employs spectral basis function projections to accurately capture high-frequency dynamics, leverages the intrinsic time-ordered structure of NODEs to rigorously enforce temporal causality, and adopts a classical spectral-method-inspired parameter initialization strategy to accelerate convergence. The model is formulated as an end-to-end differentiable framework incorporating physics-constrained loss, high-order numerical integration, and spectral learning mechanisms. Evaluated on diverse linear and nonlinear wave equation benchmarks, NeuSA achieves substantial improvements in convergence speed, temporal consistency, and prediction accuracy—demonstrating its effectiveness, robustness, and generalizability.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Physics-Informed Neural Networks (PINNs) have emerged as a powerful neural framework for solving partial differential equations (PDEs). However, standard MLP-based PINNs often fail to converge when dealing with complex initial-value problems, leading to solutions that violate causality and suffer from a spectral bias towards low-frequency components. To address these issues, we introduce NeuSA (Neuro-Spectral Architectures), a novel class of PINNs inspired by classical spectral methods, designed to solve linear and nonlinear PDEs with variable coefficients. NeuSA learns a projection of the underlying PDE onto a spectral basis, leading to a finite-dimensional representation of the dynamics which is then integrated with an adapted Neural ODE (NODE). This allows us to overcome spectral bias, by leveraging the high-frequency components enabled by the spectral representation; to enforce causality, by inheriting the causal structure of NODEs, and to start training near the target solution, by means of an initialization scheme based on classical methods. We validate NeuSA on canonical benchmarks for linear and nonlinear wave equations, demonstrating strong performance as compared to other architectures, with faster convergence, improved temporal consistency and superior predictive accuracy. Code and pretrained models will be released.

Problem

• Overcoming spectral bias in solving complex PDEs
• Enforcing causality in Physics-Informed Neural Networks
• Improving convergence for initial-value problems with variable coefficients

Innovation

• Spectral basis projection for PDE representation
• Integration with adapted Neural ODE framework
• Classical initialization for near-target training