HEATNETs: Explainable Random Feature Neural Networks for High-Dimensional Parabolic PDEs

2025-11-02

AI Summary

Solving high-dimensional parabolic partial differential equations (PDEs) remains challenging due to the curse of dimensionality and singularities in fundamental solutions. Method: We propose HEATNETs—a stochastic feature neural network grounded in the heat kernel Green’s function. It integrates the integral representation of PDE solutions with physics-informed neural networks, employs Monte Carlo importance sampling to mitigate heat kernel singularity, and achieves unbiased, interpretable, theoretically guaranteed approximation via heat kernel projection into a single hidden layer. Contribution/Results: This work introduces, for the first time, a heat-kernel-driven random feature mechanism ensuring rigorous function-space adaptivity and scalability to high dimensions. Experiments demonstrate stable convergence on linear parabolic PDEs up to 2000 dimensions: relative errors reach 10⁻⁵–10⁻⁷ within 500D, and 10⁻⁴–10⁻³ within 1000–2000D, using ≤15,000 random features—substantially improving accuracy, efficiency, and interpretability for high-dimensional PDE solving.

Technology Category

• Machine Learning: Kernel Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Heuristic Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

We deal with the solution of the forward problem for high-dimensional parabolic PDEs with random feature (projection) neural networks (RFNNs). We first prove that there exists a single-hidden layer neural network with randomized heat-kernels arising from the fundamental solution (Green's functions) of the heat operator, that we call HEATNET, that provides an unbiased universal approximator to the solution of parabolic PDEs in arbitrary (high) dimensions, with the rate of convergence being analogous to the ${O}(N^{-1/2})$, where $N$ is the size of HEATNET. Thus, HEATNETs are explainable schemes, based on the analytical framework of parabolic PDEs, exploiting insights from physics-informed neural networks aided by numerical and functional analysis, and the structure of the corresponding solution operators. Importantly, we show how HEATNETs can be scaled up for the efficient numerical solution of arbitrary high-dimensional parabolic PDEs using suitable transformations and importance Monte Carlo sampling of the integral representation of the solution, in order to deal with the singularities of the heat kernel around the collocation points. We evaluate the performance of HEATNETs through benchmark linear parabolic problems up to 2,000 dimensions. We show that HEATNETs result in remarkable accuracy with the order of the approximation error ranging from $1.0E-05$ to $1.0E-07$ for problems up to 500 dimensions, and of the order of $1.0E-04$ to $1.0E-03$ for 1,000 to 2,000 dimensions, with a relatively low number (up to 15,000) of features.

Problem

• Solving high-dimensional parabolic PDEs using explainable neural networks
• Providing unbiased universal approximators for parabolic PDE solutions
• Scaling neural networks for efficient high-dimensional PDE computation

Innovation

• Random feature neural networks with heat-kernel basis functions
• Importance Monte Carlo sampling for high-dimensional integrals
• Unbiased universal approximator for parabolic PDEs