Efficient Bayesian Deep Ensembles via Analytic Predictive Inference

2026-07-07

AI Summary

This work proposes an efficient Bayesian deep ensemble method to address the limitations of existing deep ensembles in uncertainty calibration and interpretability. By leveraging low-dimensional predictive representations, independent training strategies, and a closed-form Bayesian linear regression aggregation mechanism, the approach enables analytical posterior weight inference while maintaining high predictive accuracy. This significantly enhances model interpretability and yields well-calibrated uncertainty estimates. Notably, the computational complexity shifts from scaling with dataset size to depending only on ensemble size, substantially improving scalability. Empirical evaluations on standard regression benchmarks demonstrate that the method achieves state-of-the-art predictive performance while providing reliable and properly calibrated uncertainty quantification.

Abstract

We introduce an efficient Bayesian deep ensemble method for predictive regression designed to enhance interpretability while maintaining competitive predictive performance and computational efficiency. Our method combines the statistical rigor of Bayesian inference with the scalability of deep ensembles, providing calibrated uncertainty estimates that enable its use not only for standalone prediction but also as a component within broader learning systems. To achieve these goals, our work relies on three key design components: (i) low-dimensional ensemble representation: predictions are expressed as a combination of a small number of trained neural predictors, enabling scalable inference whose cost depends on ensemble size rather than dataset size; (ii) closed-form Bayesian aggregation: ensemble predictions are combined using Bayesian linear regression, yielding interpretable posterior weights and calibrated uncertainty without approximate inference; and (iii) Independent ensemble training: multiple neural networks are trained separately, producing diverse predictive representations that improve robustness and uncertainty calibration. Empirical results on standard regression benchmarks demonstrate that the proposed approach achieves competitive predictive performance while maintaining reliable uncertainty estimates across settings.

Problem

• Bayesian deep ensembles
• predictive regression
• uncertainty calibration
• interpretability
• computational efficiency

Innovation

• Bayesian deep ensembles
• analytic predictive inference
• uncertainty calibration
• closed-form Bayesian aggregation
• low-dimensional ensemble representation