Even More Guarantees for Variational Inference in the Presence of Symmetries

2026-04-23

AI Summary

This work demonstrates that variational inference can still exactly recover the mean of a target distribution even when the variational family fails to cover the true posterior, provided the target exhibits certain symmetries. Focusing on forward KL divergence and α-divergences, the study establishes the first sufficient conditions under which the posterior mean remains precisely identifiable despite model misspecification, thereby elucidating the mechanisms underlying optimization failure in such settings. By integrating tools from information geometry, symmetry analysis, and moment recovery theory, this research not only extends theoretical guarantees for robust variational inference under symmetry but also offers concrete guidance for selecting both the variational family and the α parameter in practical applications.

Technology Category

• Reasoning under Uncertainty: Probabilistic Inference
• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems

Abstract

When approximating an intractable density via variational inference (VI) the variational family is typically chosen as a simple parametric family that very likely does not contain the target. This raises the question: Under which conditions can we recover characteristics of the target despite misspecification? In this work, we extend previous results on robust VI with location-scale families under target symmetries. We derive sufficient conditions guaranteeing exact recovery of the mean when using the forward Kullback-Leibler divergence and $α$-divergences. We further show how and why optimization can fail to recover the target mean in the absence of our sufficient conditions, providing initial guidelines on the choice of the variational family and $α$-value.

Problem

• variational inference
• model misspecification
• symmetries
• mean recovery
• α-divergences

Innovation

• variational inference
• symmetries
• α-divergences
• robustness
• mean recovery