Shallow ReLU$^s$ Networks in $L^p$-Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization

2026-05-18

AI Summary

This work investigates the approximation capabilities of shallow ReLU$^s$ neural networks in $L^p$-type and Sobolev spaces, along with their generalization performance under path norm constraints. By leveraging spherical harmonic analysis and embeddings into spectral Barron spaces, the study establishes approximation rates for these networks in the aforementioned function spaces and analyzes the generalization behavior of nonparametric regression with path norm regularization. Under the condition $p < p^*$, the derived approximation rates improve upon those achievable by random feature methods. Most notably, this work provides the first minimax optimal generalization error bounds—matching up to logarithmic factors—for ReLU$^s$ networks in both Barron and Sobolev spaces.

Abstract

We study approximation by shallow ReLU$^s$ networks, $σ_s(t)=\max{0,t}^s$, and the generalization behavior of such networks under $\ell_1$ path-norm control. For the $L^p$-type integral spaces $\widetilde{\mathcal{F}}_{p,τ_d,s}$, $1\le p\le2$, we establish approximation bounds for shallow networks using spherical harmonic analysis. In particular, when the parameter measure is the uniform measure $τ_d$ and $p<p^*=(2d+2)/(d+3)$, we obtain the rate $O(m^{-1/2-d(2-p)/(2d(2-p)+2p(2s+d+1))}\log^{3/2}m)$, which improves the corresponding random-feature rate. We also derive approximation rates for Sobolev spaces $W^{α,p}$ in the range $1\le p<2$ by embedding them into spectral Barron spaces. Finally, for nonparametric regression with sub-Gaussian noise, we prove minimax-optimal generalization bounds for path-norm-regularized shallow ReLU$^s$ networks over Barron and Sobolev spaces, with matching lower bounds up to logarithmic factors.

Problem

• shallow ReLU networks
• approximation theory
• Sobolev spaces
• generalization bounds
• path-norm regularization

Innovation

• shallow ReLU^s networks
• approximation theory
• path-norm regularization
• Sobolev spaces
• minimax optimality