🤖 AI Summary
Many physical models exhibit singularities—e.g., boundary layers or $|x|^alpha$-type behavior in fracture mechanics—where standard neural networks, constrained by fixed activation functions, suffer from poor approximation efficiency. To address this, we propose a learnable power-law basis network, the first architecture to explicitly embed the Müntz–Szász approximation theorem. Its activation function, $varphi(x)=sum a_k|x|^{mu_k}+sum b_koperatorname{sign}(x)|x|^{lambda_k}$, features jointly learnable exponents $mu_k,lambda_k$ and coefficients. We prove its universal approximation property and derive a squared-order error bound strictly superior to that of multilayer perceptrons (MLPs). Experiments demonstrate: (i) 5–8× lower error and 10× fewer parameters on singular regression tasks; (ii) 3–6× higher accuracy on physics-informed neural network (PINN) benchmarks; and (iii) learned exponents that quantitatively recover the true singularity structure of analytical solutions.
📝 Abstract
Standard neural network architectures employ fixed activation functions (ReLU, tanh, sigmoid) that are poorly suited for approximating functions with singular or fractional power behavior, a structure that arises ubiquitously in physics, including boundary layers, fracture mechanics, and corner singularities. We introduce Müntz-Szász Networks (MSN), a novel architecture that replaces fixed smooth activations with learnable fractional power bases grounded in classical approximation theory. Each MSN edge computes $φ(x) = sum_k a_k |x|^{μ_k} + sum_k b_k mathrm{sign}(x)|x|^{λ_k}$, where the exponents ${μ_k, λ_k}$ are learned alongside the coefficients. We prove that MSN inherits universal approximation from the Müntz-Szász theorem and establish novel approximation rates: for functions of the form $|x|^α$, MSN achieves error $mathcal{O}(|μ- α|^2)$ with a single learned exponent, whereas standard MLPs require $mathcal{O}(ε^{-1/α})$ neurons for comparable accuracy. On supervised regression with singular target functions, MSN achieves 5-8x lower error than MLPs with 10x fewer parameters. Physics-informed neural networks (PINNs) represent a particularly demanding application for singular function approximation; on PINN benchmarks including a singular ODE and stiff boundary-layer problems, MSN achieves 3-6x improvement while learning interpretable exponents that match the known solution structure. Our results demonstrate that theory-guided architectural design can yield dramatic improvements for scientifically-motivated function classes.