Testing (Conditional) Mutual Information

2025-06-04

AI Summary

This paper addresses the hypothesis testing problem for conditional mutual information (CMI): given i.i.d. samples from a triple (A, B, C), distinguish H₀: I(A; C | B) = 0 from H₁: I(A; C | B) ≥ ε. We propose the first CMI test with a tight sample complexity upper bound of O(1/ε²), proven minimax optimal (up to constant factors) in the multi-parameter regime. Our method introduces a novel nonparametric CMI estimator based on weakly correlated sample simulation and embeds it within an equivalence testing framework tailored to CMI. The theoretical analysis integrates tools from information theory, distribution approximation, and nonparametric statistics; as a byproduct, our marginal mutual information (MI) test achieves optimal sample complexity (up to polylogarithmic factors). The key contribution is breaking the long-standing sample-efficiency bottleneck in CMI testing, thereby establishing a rigorous and practically applicable statistical foundation for high-dimensional conditional independence inference.

Technology Category

• Machine Learning: Information Theory
• Reasoning under Uncertainty: Relational Probabilistic Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Web measurements

Abstract

We investigate the sample complexity of mutual information and conditional mutual information testing. For conditional mutual information testing, given access to independent samples of a triple of random variables $(A, B, C)$ with unknown distribution, we want to distinguish between two cases: (i) $A$ and $C$ are conditionally independent, i.e., $I(A!:!C|B) = 0$, and (ii) $A$ and $C$ are conditionally dependent, i.e., $I(A!:!C|B) geq varepsilon$ for some threshold $varepsilon$. We establish an upper bound on the number of samples required to distinguish between the two cases with high confidence, as a function of $varepsilon$ and the three alphabet sizes. We conjecture that our bound is tight and show that this is indeed the case in several parameter regimes. For the special case of mutual information testing (when $B$ is trivial), we establish the necessary and sufficient number of samples required up to polylogarithmic terms. Our technical contributions include a novel method to efficiently simulate weakly correlated samples from the conditionally independent distribution $P_{A|B} P_{C|B} P_B$ given access to samples from an unknown distribution $P_{ABC}$, and a new estimator for equivalence testing that can handle such correlated samples, which might be of independent interest.

Problem

• Sample complexity of mutual information testing
• Conditional independence testing between variables
• Upper bound on samples for dependency detection

Innovation

• Novel sample simulation for conditional independence
• New estimator for equivalence testing
• Tight sample complexity bounds established