Conditions for equality and stability in Shannon's and Tao's entropy power inequalities

2025-09-17

AI Summary

This work systematically characterizes the necessary and sufficient conditions for equality and the stability mechanisms of both Shannon’s entropy power inequality (EPI) and Tao’s discrete EPI. Methodologically, it establishes the de Bruijn identity without assuming finite second moments—achieved via weak convergence analysis, compactness arguments, the Cheeger inequality, and concentration properties of discrete log-concave distributions—while reconstructing the Bobkov–Chistyakov counterexample. The theoretical contributions are threefold: (1) a rigorous proof that equality in Shannon’s EPI holds if and only if the random variables are Gaussian; (2) a qualitative stability result within a weak convergence framework; and (3) the first quantitative stability bound for discrete log-concave distributions. Collectively, these results unify and reinforce the central role of Gaussianity, substantially extending the applicability and robustness of the EPI beyond classical regularity assumptions.

Technology Category

• Machine Learning: Information Theory
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We show that there is equality in Shannon's Entropy Power Inequality (EPI) if and only if the random variables involved are Gaussian, assuming nothing beyond the existence of differential entropies. This is done by justifying de Bruijn's identity without a second moment assumption. Part of the proof also relies on a re-examination of an example of Bobkov and Chistyakov (2015), which shows that there exists a random variable $X$ with finite differential entropy $h(X),$ such that $h(X+Y) = infty$ for any independent random variable $Y$ with finite entropy. We prove that either $X$ has this property, or $h(X+Y)$ is finite for any independent $Y$ that does not have this property. Using this, we prove the continuity of $t mapsto h(X+sqrt{t}Z)$ at $t=0$, where $Z sim mathcal{N}(0,1)$ is independent of $X$, under minimal assumptions. We then establish two stability results: A qualitative stability result for Shannon's EPI in terms of weak convergence under very mild moment conditions, and a quantitative stability result in Tao's discrete analogue of the EPI under log-concavity. The proof for the first stability result is based on a compactness argument, while the proof of the second uses the Cheeger inequality and leverages concentration properties of discrete log-concave distributions.

Problem

• Characterizing equality conditions for Shannon's Entropy Power Inequality
• Establishing stability results for Shannon's EPI under weak convergence
• Providing quantitative stability for Tao's discrete EPI under log-concavity

Innovation

• Proves EPI equality only for Gaussian variables
• Establishes continuity of entropy under minimal assumptions
• Provides stability results using compactness and Cheeger