The Binomial Channel: On Capacity, Optimal Inputs, and Beta-Binomial Approximation

2026-07-02

AI Summary

This study investigates the capacity of a binary channel with continuous input and discrete output, along with the structure of its optimal input distribution. Through information-theoretic analysis, convex optimization, and Beta-Binomial modeling—augmented by minimax redundancy constructions and divergence measures including relative entropy and χ² divergence—the authors prove that the optimal input distribution is discrete, unique, symmetric about 1/2, and includes the endpoints of the interval. Key contributions include tightening the upper bound on the support size of the optimal input from O(n) to O(n/2) and establishing the first lower bound of Ω(√(n log log n)). They further show that the output distribution induced by Beta(1/2,1/2) is asymptotically optimal, derive non-asymptotic capacity bounds, and establish that C(n) = ½ log(nπ/(2e)) + o(1), with numerical experiments corroborating the theoretical findings.

Abstract

We study the binomial channel with input alphabet $[0,1]$ and output alphabet ${0,\ldots,n}$. We investigate its capacity and the structure of the capacity-achieving input and output distributions. Since the output alphabet is finite whereas the input alphabet is continuous, different input distributions may induce the same output distribution; hence, uniqueness and support properties of optimal inputs do not follow from strict concavity arguments. We first establish structural properties of the capacity-achieving input distribution. In particular, we show that it is discrete, unique, symmetric around $1/2$, and contains the endpoints ${0,1}$ in its support. We also derive location constraints and bounds on the probability masses of support points, and improve the Witsenhausen-type upper bound on the support size from order $n$ to order $n/2$. We derive explicit nonasymptotic upper and lower bounds on the capacity $C(n)$. These bounds imply $C(n)=\frac{1}{2}\log\frac{nπ}{2e}+o(1).$ The lower bound is obtained by evaluating the mutual information at the reference input $X_r\sim \mathrm{Beta}(1/2,1/2)$, which induces a beta-binomial output distribution, while the upper bound follows from a minimax redundancy construction. Finally, we prove an improved lower bound on the support size of the capacity-achieving input distribution. We show that the beta-binomial output induced by $X_r$ is asymptotically optimal and close to the capacity-achieving output distribution in relative entropy and $χ^2$ divergence. We also prove a finite-mixture approximation lower bound showing that the beta-binomial output cannot be approximated too accurately by binomial mixtures with few components. Combining these results yields a support-size lower bound of order $Ω(\sqrt{n\log\log n})$, with explicit constants. Numerical results illustrate the capacity bounds and optimal input distribution.

Problem

• binomial channel
• channel capacity
• optimal input distribution
• beta-binomial approximation
• support size

Innovation

• binomial channel
• capacity-achieving input
• beta-binomial approximation
• support size bounds
• mutual information