Adaptive-Distribution Randomized Neural Networks for PDEs: A Low-Dimensional Distribution-Learning Framework

2026-04-26

AI Summary

This work addresses the sensitivity of random neural networks for solving partial differential equations (PDEs) to the sampling distribution of hidden-layer parameters, a limitation in conventional approaches that rely on heuristic design and suffer from poor stability. The authors propose the AD-RaNN framework, which reformulates distribution selection as an optimizable low-dimensional parameter problem. By preserving the linear least-squares structure while incorporating a PDE- and data-driven adaptive distribution mechanism alongside an adaptive layer-growing strategy, the method reduces dependence on manual tuning. Leveraging low-dimensional parameterization, ridge-regularized optimization, efficient gradient computation, and integration of multiple solvers, AD-RaNN achieves stable, adaptive, high-accuracy solution recovery and strong generalization across multiple benchmark problems.

Technology Category

• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising

Abstract

Randomized neural networks (RaNNs) are attractive for partial differential equations (PDEs) because they replace expensive end-to-end training with a linear least-squares solve over randomized hidden features. Their practical performance, however, depends strongly on the sampling distribution of the hidden-layer parameters, which is usually chosen heuristically and problem by problem. This distribution sensitivity is a central bottleneck in randomized neural PDE solvers. In this work, we propose Adaptive-Distribution Randomized Neural Networks (AD-RaNN), a framework that promotes randomized feature generation from a fixed heuristic choice to a low-dimensional adaptive optimization problem. Instead of training all hidden weights and biases, AD-RaNN parameterizes the hidden-feature sampling distribution by a low-dimensional vector p and optimizes only p, thereby preserving the least-squares structure of RaNNs while reducing manual distribution tuning. The method uses a two-stage strategy: ridge-regularized reduced training for stable distribution-parameter optimization, followed by an unregularized least-squares refit for final solution recovery. We develop two adaptive mechanisms, PDE-Driven Adaptive Distribution (PDAD) and Data-Driven Adaptive Distribution (DDAD), and deploy them in space-time solvers, discrete-time solvers, and operator-learning models. We also incorporate an adaptive layer-growth enhancement for localized structures. For the reduced optimization problem, we establish well-posedness of the reduced objectives, consistency of ridge-regularized minimizers, an efficient gradient formula, and a practical lower-bound estimate for the ridge parameter. Numerical experiments on benchmark problems show that AD-RaNN provides an effective distribution-level adaptation mechanism, reduces reliance on hand-crafted hidden-feature distributions, and achieves strong empirical accuracy.

Problem

• Randomized Neural Networks
• Partial Differential Equations
• Sampling Distribution
• Distribution Sensitivity
• Hidden-Feature Distribution

Innovation

• adaptive distribution
• randomized neural networks
• low-dimensional optimization
• PDE solvers
• distribution learning