Measuring Dependence between Events

2024-03-26

AI Summary

Existing dependence measures for binary events—such as the phi coefficient—lack a rigorous axiomatic foundation and frequently conflate event equality with statistical dependence, leading to severe underestimation of true association. Method: We establish the first axiomatic framework for dependence strength in 2×2 contingency tables, formalizing four natural, interpretable axioms grounded in information-theoretic and probabilistic principles. We derive exact asymptotic distributions for major measures and construct corresponding confidence intervals. Contribution/Results: We prove that Yule’s Q and the Cole coefficient are the *only* widely adopted measures satisfying all four axioms; in contrast, phi and others suffer from a fundamental “reachability deficiency” that violates monotonicity under marginal-preserving dependence strengthening. Empirical evaluation on real-world pharmacovigilance data demonstrates that misusing conventional metrics yields statistically significant distortions in inferred drug-consumption associations. This work provides both a theoretical criterion for evaluating dependence measures and practical guidance for their principled application in binary contingency analysis.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Preferences
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

Measuring dependence between two events, or equivalently between two binary random variables, amounts to expressing the dependence structure inherent in a $2 imes 2$ contingency table in a real number between $-1$ and $1$. Countless such dependence measures exist, but there is little theoretical guidance on how they compare and on their advantages and shortcomings. Thus, practitioners might be overwhelmed by the problem of choosing a suitable measure. We provide a set of natural desirable properties that a proper dependence measure should fulfill. We show that Yule's Q and the little-known Cole coefficient are proper, while the most widely-used measures, the phi coefficient and all contingency coefficients, are improper. They have a severe attainability problem, that is, even under perfect dependence they can be very far away from $-1$ and $1$, and often differ substantially from the proper measures in that they understate strength of dependence. The structural reason is that these are measures for equality of events rather than of dependence. We derive the (in some instances non-standard) limiting distributions of the measures and illustrate how asymptotically valid confidence intervals can be constructed. In a case study on drug consumption we demonstrate how misleading conclusions may arise from the use of improper dependence measures.

Problem

• Comparing dependence measures for binary variables in contingency tables
• Identifying proper dependence measures fulfilling desirable properties
• Demonstrating misleading conclusions from improper dependence measures in practice

Innovation

• Proposes Yule's Q as proper dependence measure
• Identifies Cole coefficient as valid alternative metric
• Highlights phi coefficient limitations for binary data