A Stein Identity for q-Gaussians with Bounded Support

2026-03-03

AI Summary

Existing Stein identities are ill-suited for non-Gaussian distributions with bounded support—such as q-Gaussian distributions—limiting their applicability in gradient estimation for generative modeling and stochastic optimization. This work presents the first extension of Bonnet- and Price-type theorems to q-Gaussian distributions by introducing an auxiliary companion distribution, yielding a new Stein identity that is both analytically concise and computationally tractable. The resulting gradient estimator preserves structural similarity to its Gaussian counterpart while significantly reducing variance. Empirical evaluations demonstrate its superior performance in Bayesian deep learning and sharpness-aware minimization tasks, thereby broadening both the theoretical foundations and practical applicability of Stein’s method beyond the Gaussian setting.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Stein's identity is a fundamental tool in machine learning with applications in generative models, stochastic optimization, and other problems involving gradients of expectations under Gaussian distributions. Less attention has been paid to problems with non-Gaussian expectations. Here, we consider the class of bounded-support $q$-Gaussians and derive a new Stein identity leading to gradient estimators which have nearly identical forms to the Gaussian ones, and which are similarly easy to implement. We do this by extending the previous results of Landsman, Vanduffel, and Yao (2013) to prove new Bonnet- and Price-type theorems for q-Gaussians. We also simplify their forms by using escort distributions. Our experiments show that bounded-support distributions can reduce the variance of gradient estimators, which can potentially be useful for Bayesian deep learning and sharpness-aware minimization. Overall, our work simplifies the application of Stein's identity for an important class of non-Gaussian distributions.

Problem

• Stein identity
• q-Gaussian
• bounded support
• non-Gaussian distributions
• gradient estimation

Innovation

• Stein identity
• q-Gaussian
• bounded support
• gradient estimation
• escort distribution