Sobolev Approximation by Fixed-Size Neural Networks with Arbitrary Accuracy

2026-06-15

AI Summary

This study addresses the problem of approximating functions with arbitrary precision in Sobolev spaces using neural networks of fixed architecture. To this end, the authors introduce novel differentiable universal activation functions—EUAF and the DUAF family, including DUAF_∞ and its sigmoid variant—which for the first time enable fixed-structure neural networks to achieve Sobolev approximation in the \(W^{s,\infty}\) space. Theoretically, they prove that networks employing these activation functions, with both width and depth held constant, can approximate any function in \(W^{s,\infty}\) arbitrarily well under the \(W^{s-1,\infty}\) norm. Moreover, explicit bounds on the network parameters are provided, thereby overcoming the conventional limitation that network size must increase with desired approximation accuracy.

Abstract

In this work, we investigate new activation functions for achieving arbitrary-accuracy Sobolev approximation by fixed-size neural networks. We first show that any function in $W^{2,\infty}((a,b)^d)$ can be approximated with arbitrary accuracy, measured in the $W^{1,\infty}$-norm, by a fixed-size neural network using the Elementary Universal Activation Function ($\mathrm{EUAF}$). To extend this result to $W^{s,\infty}((a,b)^d)$ for $s\in\mathbb{N}$, we introduce a smooth activation $\mathrm{DUAF}_{\infty}$ from the family of Differentiable Universal Activation Functions ($\mathrm{DUAF}_n$). We prove that any function in $W^{s,\infty}((a,b)^d)$ can be approximated with arbitrary accuracy in the $W^{s-1,\infty}$-norm by a fixed-size $\mathrm{DUAF}_{\infty}$-activated network. We further construct sigmoidal variants $\widetilde{\mathrm{DUAF}}_n$ and show that, for every $1\leq s\leq n$, fixed-size $\widetilde{\mathrm{DUAF}}_n$-activated networks still approximate any $f\in W^{s,\infty}((a,b)^d)$ with arbitrary accuracy in the $W^{s-1,\infty}$-norm. In all these results, the width and depth bounds are computed explicitly, and the proposed activations are elementary.

Problem

• Sobolev approximation
• fixed-size neural networks
• arbitrary accuracy
• activation functions
• function approximation

Innovation

• Sobolev approximation
• fixed-size neural networks
• universal activation functions
• arbitrary accuracy
• DUAF