Beyond Mutual Information: Extension Profiles and Shape Functions of Random Variable Pairs

2026-06-22

AI Summary

Traditional mutual information fails to fully capture the structural properties of joint distributions. This work introduces an auxiliary variable \(W\) and proposes the extended profile and its shape function for a pair of random variables \((X,Y)\), applying them for the first time to the structural analysis of joint distributions. By characterizing the profile boundary via the Legendre–Fenchel transform and integrating tools from biregular bipartite graph modeling, spectral graph theory, and information theory, the study uncovers deep connections among non-Shannon-type inequalities, Gács–Körner common information, and graph spectra. Furthermore, it establishes general upper and lower bounds on the shape function and, for variable pairs with uniformly distributed support sets, derives tight bounds based on the second-largest eigenvalue of the associated graph, thereby delineating the set of achievable extended types.

Abstract

We study the extension profile of a pair of jointly distributed finite-valued random variables $(X,Y)$, defined as the set of all triples of numbers $ (H(X|W), H(Y|W), I(X:Y|W)) $ obtained by extending the pair with an auxiliary random variable $W$. This object captures structural properties of joint distributions that are not determined solely by the entropies of $X$ and $Y$ and their mutual information. To describe the boundary of the extension profile, we introduce the associated shape function, defined as the Legendre--Fenchel transform of the nontrivial part of the profile boundary. We establish general upper and lower bounds on the shape function in terms of classical information-theoretic quantities. For pairs that are uniform on their support, we interpret the support as a biregular bipartite graph and relate the extension profile to combinatorial and spectral properties of this graph. In this setting, we derive bounds on the shape function in terms of the second-largest eigenvalue of the graph. Thus, pairs whose support graphs have a small second eigenvalue admit only a restricted class of extensions. Our results provide a new perspective on the information-theoretic structure of joint distributions and highlight connections among non-Shannon-type information inequalities, the Gács--Körner common information, and spectral graph theory. We discuss several applications of the developed framework to problems concerning the structure and representation of mutual information.

Problem

• extension profile
• shape function
• mutual information
• joint distribution
• common information

Innovation

• extension profile
• shape function
• spectral graph theory
• non-Shannon inequalities
• Gács–Körner common information