The Blahut--Arimoto Algorithm as a Dynamical System with Exact $χ^2$ Dissipation

2026-04-27

AI Summary

This work uncovers the global dissipative mechanism and information-geometric structure of the Blahut–Arimoto (BA) algorithm in continuous time. By modeling the BA flow as a dynamical system driven by the Pearson $\chi^2$ divergence, it establishes for the first time that this divergence serves as the global non-asymptotic driving force and demonstrates an exact correspondence between the BA flow and a free energy gradient flow. Furthermore, it reveals that the symmetric Jeffreys divergence differs from the $\chi^2$ divergence only by a third-order remainder term that vanishes identically along the BA flow, thereby unifying several information divergences algebraically. Integrating tools from information geometry, dynamical systems, and spectral analysis, the study derives a closed-form ODE for Gaussian sources, proving the existence of a unique attractor and a spectral rigidity phenomenon governing high-dimensional convergence—offering a novel dissipative perspective on MIMO capacity, Wyner–Ziv coding, and the information bottleneck principle.

Abstract

This paper uncovers an exact $χ^2$ dissipation identity for the Blahut--Arimoto (BA) flow and establishes its fundamental information-geometric structure. While prior works have analyzed BA convergence via the monotone decrease of the Kullback--Leibler divergence, we establish that the continuous-time flow $\partial_τq = \tilde{q}_β[q] - q$ is fundamentally governed by the Pearson $χ^2$ divergence. Specifically, we prove the governing identity $\frac{d}{dτ} Φ_β= -χ^2(\tilde{q}_β\| q)$, elevating $χ^2$ from a mere local approximation to the global non-asymptotic driver of the dynamics. We establish a \emph{unified dissipation identity} revealing that the $χ^2$ and the symmetrised Jeffreys divergence $D_J$ differ only by a cubic residual $\mathcal{N}$, which vanishes \emph{exactly} for the BA flow. This algebraic cancellation identifies the Jacobian $K_q = D\tilde{q}_β[q]$ and its complement $\mathcal{A}_q = I - K_q$ as the Fisher--Rao Hessian of the free energy $Φ_β$. The spectral gap of $\mathcal{A}_{q^*}$ is proved to determine both the local exponential convergence rate (a universal value of $2$ in artificial time) and the stability of the attraction basin. For Gaussian sources, the theory yields three concrete consequences: (i)~the reproduction variance obeys an exact closed ODE; (ii)~the Gaussian distribution emerges as the unique dynamical attractor; and (iii)~in high dimensions, convergence is bottlenecked by the maximum source variance -- \emph{spectral stiffness}. Significantly, this framework provides new dissipative interpretations for fundamental problems including MIMO channel capacity, Wyner--Ziv coding, and the information bottleneck. These results establish the BA flow as a canonical dissipative relaxation and provide a geometric foundation for the dynamical theory of information-theoretic optimisation.

Problem

• Blahut–Arimoto algorithm
• χ² dissipation
• information geometry
• dynamical system
• convergence analysis

Innovation

• Blahut–Arimoto algorithm
• χ² dissipation
• information geometry
• dynamical system
• Fisher–Rao metric