Student-t processes as infinite-width limits of posterior Bayesian neural networks

2025-02-06

AI Summary

Standard Gaussian process (GP) approximations fail to capture heavy-tailed uncertainty in Bayesian neural networks (BNNs) under the infinite-width limit. Method: The authors rigorously analyze the asymptotic behavior of BNN posteriors, assuming Gaussian–inverse-Gamma priors over weights. Contribution/Results: They establish, for the first time, that the BNN posterior converges to a Student-t process (TP)—not a GP—in the infinite-width limit. This convergence is quantified in the Wasserstein metric, with an explicit convergence rate derived. The result formally establishes asymptotic equivalence between BNN posteriors and TPs, providing a theoretically grounded, robust, and flexible framework for uncertainty quantification in deep Bayesian models. Crucially, the TP approximation enhances resilience to outliers and distributional shift compared to GP-based alternatives.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

The asymptotic properties of Bayesian Neural Networks (BNNs) have been extensively studied, particularly regarding their approximations by Gaussian processes in the infinite-width limit. We extend these results by showing that posterior BNNs can be approximated by Student-t processes, which offer greater flexibility in modeling uncertainty. Specifically, we show that, if the parameters of a BNN follow a Gaussian prior distribution, and the variance of both the last hidden layer and the Gaussian likelihood function follows an Inverse-Gamma prior distribution, then the resulting posterior BNN converges to a Student-t process in the infinite-width limit. Our proof leverages the Wasserstein metric to establish control over the convergence rate of the Student-t process approximation.

Problem

• Student-t processes approximation
• Bayesian Neural Networks convergence
• Infinite-width limit analysis

Innovation

• Student-t processes approximation
• Infinite-width Bayesian Neural Networks
• Wasserstein metric convergence control