Variational Learning of Fractional Posteriors

2026-03-28

AI Summary

This work addresses the limitations of traditional variational inference, which struggles to calibrate posterior distributions due to the representational constraints of the evidence lower bound (ELBO). The authors propose a novel single-parameter variational objective that introduces, for the first time, an adjustable score-based posterior, thereby establishing a flexible variational framework capable of jointly learning hierarchical structures and Bayesian posteriors. By incorporating analytically tractable gradient computation, the method significantly improves posterior calibration in mixture models and achieves higher ELBO values in variational autoencoders (VAEs). This enhancement promotes better alignment between the decoder and the prior distribution, ultimately strengthening the model's probabilistic representation capabilities.

Abstract

We introduce a novel one-parameter variational objective that lower bounds the data evidence and enables the estimation of approximate fractional posteriors. We extend this framework to hierarchical construction and Bayes posteriors, offering a versatile tool for probabilistic modelling. We demonstrate two cases where gradients can be obtained analytically and a simulation study on mixture models showing that our fractional posteriors can be used to achieve better calibration compared to posteriors from the conventional variational bound. When applied to variational autoencoders (VAEs), our approach attains higher evidence bounds and enables learning of high-performing approximate Bayes posteriors jointly with fractional posteriors. We show that VAEs trained with fractional posteriors produce decoders that are better aligned for generation from the prior.

Problem

• variational inference
• fractional posteriors
• evidence lower bound
• posterior calibration
• variational autoencoders

Innovation

• fractional posteriors
• variational inference
• evidence lower bound
• hierarchical modeling
• variational autoencoders