Explicit integral representations and quantitative bounds for two-layer ReLU networks

2026-04-25

AI Summary

This work investigates the construction of explicit integral representations for two-layer ReLU neural networks and establishes L² approximation error bounds that do not explicitly depend on the input dimension or polynomial degree. By leveraging harmonic extension and projection operators, the authors derive a simplified integral representation of ReLU networks capable of approximating arbitrary multivariate polynomial functions. The resulting error bound is determined solely by the monomial coefficients and the underlying data distribution, thereby eliminating the conventional dependence on dimensionality and polynomial order. This advancement significantly enhances approximation efficiency and theoretical scalability in high-dimensional settings.

Technology Category

• Machine Learning: Representation Learning
• Computer Vision: Representation Learning for Vision
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

An approach to construct explicit integral representations for two-layer ReLU networks is presented, which provides relatively simple representations for any multivariate polynomial. Quantitative bounds are provided for a particular, sharpened ReLU integral representation, which involves a harmonic extension and a projection. The bounds demonstrate that functions can be approximated with $L^{2}(\mathcal{D})$ errors that do not depend explicitly on dimension or degree, but rather the coefficients of their monomial expansions and the distribution $\mathcal{D}$.

Problem

• ReLU networks
• integral representations
• quantitative bounds
• function approximation
• dimension independence

Innovation

• ReLU networks
• integral representation
• harmonic extension
• quantitative bounds
• dimension-independent approximation