Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionality

2025-06-23

AI Summary

This paper investigates posterior contraction rates of sparse Bayesian neural networks for estimating functions residing in anisotropic Besov spaces—encompassing additive, multiplicative, and other hierarchical structures—to mitigate the curse of dimensionality in high-dimensional function estimation. Methodologically, it introduces, for the first time, a coupling of sparse priors with continuous shrinkage priors within a Bayesian neural network framework, enabling adaptive theoretical analysis without requiring prior knowledge of the true function’s smoothness or intrinsic dimension. Key contributions are: (1) rigorous proof that such networks automatically adapt to the underlying low-dimensional structure of the target function; (2) derivation of minimax-optimal posterior contraction rates depending solely on the intrinsic dimension, not the ambient dimension; and (3) the first rigorous, adaptive, and broadly applicable theoretical foundation for Bayesian nonparametric inference on structured high-dimensional functions.

Technology Category

• Machine Learning: Bayesian Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

This work establishes that sparse Bayesian neural networks achieve optimal posterior contraction rates over anisotropic Besov spaces and their hierarchical compositions. These structures reflect the intrinsic dimensionality of the underlying function, thereby mitigating the curse of dimensionality. Our analysis shows that Bayesian neural networks equipped with either sparse or continuous shrinkage priors attain the optimal rates which are dependent on the intrinsic dimension of the true structures. Moreover, we show that these priors enable rate adaptation, allowing the posterior to contract at the optimal rate even when the smoothness level of the true function is unknown. The proposed framework accommodates a broad class of functions, including additive and multiplicative Besov functions as special cases. These results advance the theoretical foundations of Bayesian neural networks and provide rigorous justification for their practical effectiveness in high-dimensional, structured estimation problems.

Problem

• Optimal posterior contraction rates for sparse Bayesian neural networks
• Mitigating curse of dimensionality via intrinsic dimensionality
• Rate adaptation for unknown smoothness levels

Innovation

• Sparse Bayesian networks optimize posterior contraction
• Shrinkage priors adapt to unknown smoothness levels
• Framework supports high-dimensional structured estimation