Leveraging tails for adaptation

2026-06-18

AI Summary

This study investigates how the tail behavior of nonparametric Bayesian priors influences their ability to adaptively estimate the smoothness of an unknown function. By analyzing priors with polynomially decaying tails—indexed by exponent \( p \), encompassing Laplace and heavier-tailed distributions—the authors theoretically demonstrate that posterior contraction rates improve as \( p \) decreases, achieving near-optimal smoothness adaptation in the limit as \( p \to 0 \). The work further establishes, for the first time, that over-parameterized shallow ReLU neural networks can adapt to any regularity level \( 0 \leq \beta \leq 2 \) under both white noise regression and random design settings. Numerical experiments corroborate these theoretical findings, illustrating superior performance across a broad range of smoothness classes.

Abstract

We consider contraction of Bayesian posterior distributions in nonparametric settings where coefficients of a function over a basis or dictionary are given priors with $p$--exponential tails, including Laplace tails $(p=1)$ and heavier tails $(p<1)$. It is shown that contraction rates improve as $p$ decreases and that full adaptation to smoothness, up to logarithmic factors, is obtained in an appropriate $p\to 0$ regime. As applications, we consider both series priors in white noise regression and shallow ReLU neural networks in random design regression. In particular, we show that overparametrised shallow ReLU networks can adapt to any regularity $0\le β\le 2$. Through a simulation study, we show strong empirical agreement with the behavior predicted by our theory.

Problem

• Bayesian posterior contraction
• nonparametric adaptation
• p-exponential tails
• smoothness adaptation
• ReLU neural networks

Innovation

• p-exponential tails
• posterior contraction
• adaptation to smoothness
• shallow ReLU networks
• nonparametric Bayesian inference