Beyond Global Divergences: A Local-Mass Perspective on Bayesian Inference

2026-06-25

AI Summary

This work addresses the limitation of traditional Bayesian inference, which relies on global divergences such as the Kullback–Leibler (KL) divergence and fails to capture the local quality behavior of posterior distributions. The authors propose a quality exponent and a regularized extended KL divergence (RE-KL), enabling the first systematic analysis of polynomial and logarithmic decay rates of local small-ball mass under Bayesian updating. Their framework explicitly models parameter-dependent support sets and singular components. On the theoretical front, they establish absolute, relative, and directional inequalities characterizing local mass variation. Empirical results demonstrate that the proposed approach effectively controls local posterior behavior, offering a novel localized analytical framework for Bayesian inference in high-dimensional or singular settings.

Abstract

Global objectives, such as KL divergence and ELBO, are widely used in Bayesian inference for measuring distributional discrepancy. This paper studies their local-mass behaviour that is not directly captured by such objectives. We introduce and use two mathematical tools: (1) Mass Index for recording the polynomial and logarithmic decay scales of local mass, and (2) regularised extended KL (RE-KL), a set-localised divergence that can be formulated in the presence of singular components. Mass Indices help characterise how Bayesian updating changes local mass: (1) power-log likelihood factors shift it explicitly, and (2) parameter-dependent supports, or their smooth softenings, may change the local scale through the amount of mass that remains near the parameter value. Using local RE-KL, we prove absolute, relative, and directional inequalities for comparing local small-ball masses under the two KL directions. Together, these results provide a local theoretical account of local mass behaviour. Experiments provide controlled illustrations of the local behaviour. Code is available at https://github.com/Forsythia0604/Local-Mass-Framework.

Problem

• Bayesian inference
• local mass
• KL divergence
• distributional discrepancy
• small-ball mass

Innovation

• Local-Mass
• Mass Index
• Regularised Extended KL
• Bayesian Inference
• Singular Components