Sobolev Approximation of Deep ReLU Networks in Log-Barron Space

2026-01-03

arXiv.org

AI Summary

This work addresses the limitation of the classical Barron space, whose stringent regularity requirements fail to adequately explain the remarkable approximation efficiency of deep neural networks in high dimensions. To overcome this, the authors introduce a logarithmically weighted Barron space with strictly weaker regularity conditions and establish its embedding relations with Sobolev spaces. Leveraging Rademacher complexity and function space embedding theory, they derive the first explicit approximation bounds for deep ReLU networks within this weaker regularity framework. The analysis demonstrates that such networks can efficiently approximate target functions in the $H^1$ norm and identifies the maximal effective network depth that preserves the optimal approximation rate.

Technology Category

• Machine Learning: Deep Learning Theory
• Search and Optimization: Non-convex Optimization
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Universal approximation theorems show that neural networks can approximate any continuous function; however, the number of parameters may grow exponentially with the ambient dimension, so these results do not fully explain the practical success of deep models on high-dimensional data. Barron space theory addresses this: if a target function belongs to a Barron space, a two-layer network with $n$ parameters achieves an $O(n^{-1/2})$ approximation error in $L^2$. Yet classical Barron spaces $\mathscr{B}^{s+1}$ still require stronger regularity than Sobolev spaces $H^s$, and existing depth-sensitive results often assume constraints such as $sL \le 1/2$. In this paper, we introduce a log-weighted Barron space $\mathscr{B}^{\log}$, which requires a strictly weaker assumption than $\mathscr{B}^s$ for any $s>0$. For this new function space, we first study embedding properties and carry out a statistical analysis via the Rademacher complexity. Then we prove that functions in $\mathscr{B}^{\log}$ can be approximated by deep ReLU networks with explicit depth dependence. We then define a family $\mathscr{B}^{s,\log}$, establish approximation bounds in the $H^1$ norm, and identify maximal depth scales under which these rates are preserved. Our results clarify how depth reduces regularity requirements for efficient representation, offering a more precise explanation for the performance of deep architectures beyond the classical Barron setting, and for their stable use in high-dimensional problems used today.

Problem

• Barron space
• Sobolev approximation
• deep ReLU networks
• high-dimensional approximation
• regularity requirement

Innovation

• log-weighted Barron space
• deep ReLU networks
• Sobolev approximation
• Rademacher complexity
• depth-dependent approximation