🤖 AI Summary
This work studies the average differential entropy of a $q$-ary Gaussian mixture model in $mathbb{R}^n$, where each component has isotropic covariance $sigma^2 mathbf{I}$ and mean vectors ${mathbf{W}_i}_{i=1}^q$ drawn i.i.d. from $mathcal{N}(0, s^2 mathbf{I})$. Using the scale ratio $mu = s^2/sigma^2$ as the expansion parameter, we derive a second-order asymptotic expansion (up to $O(mu^2)$) of the entropy in $mu$, and provide a general construction for higher-order terms. Methodologically, we integrate Gaussian integral techniques, random matrix analysis, and asymptotic series theory. Our key contribution is the first analytical approximation of this entropy with a rigorous, quantifiable error bound—specifically, a controllable remainder term whose order is explicitly characterized—thereby overcoming the absence of error analysis in prior work. The resulting closed-form approximation is both analytically tractable and precision-controllable, making it suitable for high-dimensional information-theoretic analysis and Gaussian mixture modeling.
📝 Abstract
We calculate the average differential entropy of a q-component Gaussian mixture in Rn. For simplicity, all components have covariance matrix σ21, while the means {Wi}i=1q are i.i.d. Gaussian vectors with zero mean and covariance s21. We obtain a series expansion in μ=s2/σ2 for the average differential entropy up to order O(μ2), and we provide a recipe to calculate higher-order terms. Our result provides an analytic approximation with a quantifiable order of magnitude for the error, which is not achieved in previous literature.