Entropy lower bounds and sum-product phenomena

2026-04-22

AI Summary

This study addresses the entropy version of the sum-product problem in information theory by establishing lower bounds on the entropies of sums and products of independent and identically distributed random variables. The authors prove that, for any random variable \(X\), the maximum of the additive entropy \(H(X+X')\) and multiplicative entropy \(H(XX')\) admits a linear lower bound in terms of \(H(X)\). Specifically, if the additive entropy doubling is bounded by a constant, then the multiplicative entropy doubling grows at least linearly with \(H(X)\). This work presents the first sum-product inequality combining Shannon entropy and min-entropy (a special case of Rényi entropy) over finite fields, and derives a stronger form over the reals, thereby revealing a profound connection between algebraic structure and information-theoretic measures across arbitrary fields.

Technology Category

• Machine Learning: Information Theory
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Security and Privacy: Applications of cryptography
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

Various lower bounds are established for the entropy of sums, products and their combinations. First, we derive a prime-field analogue of a version of the entropy power inequality established by Tao over torsion-free groups. Next, we prove an entropy sum-product statement: For independent and identically distributed random variables $X,X'$, the maximum of ${\bf H}(X+X')$ and ${\bf H}(XX')$ is bounded below by a linear combination of the entropy and the min-entropy (Rényi entropy of order~$\infty$) of $X$. This result, obtained by bounding entropies of the form ${\bf H}\bigl( X(Y+Z)\bigr)$ from above and below, is valid over arbitrary fields $F$. Over $F={\bf R}$, a slightly stronger inequality is derived. Finally, a weak version of a purely Shannon-entropic sum-product result is developed: If the entropic additive doubling of a random variable $X$ over an arbitrary field is $O(1)$, then its multiplicative doubling is at least proportional to ${\bf H}(X)$.

Problem

• entropy lower bounds
• sum-product phenomena
• Shannon entropy
• min-entropy
• entropy doubling

Innovation

• entropy power inequality
• sum-product phenomenon
• min-entropy
• entropic doubling
• Rényi entropy