Coverage correlation: detecting singular dependencies between random variables

2025-08-08

AI Summary

This paper addresses the limitation of conventional dependence measures in detecting “singular dependence”—a phenomenon where the joint distribution concentrates on a zero-measure set under the product of marginal measures. We propose the Covering Correlation (CC), an f-divergence-based nonparametric measure that is the first singular-dependence detector with rigorous statistical interpretability and a well-characterized asymptotic null distribution. Methodologically, CC integrates Monge–Kantorovich ranks with optimal transport theory, enabling efficient nonparametric estimation and natural extension to multivariate settings. Experiments demonstrate that CC achieves high sensitivity and robustness to nonlinear, nonmonotonic, and high-dimensional singular dependence structures. Moreover, in large-scale pairwise screening tasks, CC balances strong statistical power with computational efficiency.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Reasoning under Uncertainty: Graphical Models

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Web measurements

Abstract

We introduce the coverage correlation coefficient, a novel nonparametric measure of statistical association designed to quantifies the extent to which two random variables have a joint distribution concentrated on a singular subset with respect to the product of the marginals. Our correlation statistic consistently estimates an $f$-divergence between the joint distribution and the product of the marginals, which is 0 if and only if the variables are independent and 1 if and only if the copula is singular. Using Monge--Kantorovich ranks, the coverage correlation naturally extends to measure association between random vectors. It is distribution-free, admits an analytically tractable asymptotic null distribution, and can be computed efficiently, making it well-suited for detecting complex, potentially nonlinear associations in large-scale pairwise testing.

Problem

• Detects singular dependencies between random variables
• Measures association using nonparametric coverage correlation
• Evaluates joint distribution concentration on singular subsets

Innovation

• Nonparametric measure for singular dependencies
• Uses Monge-Kantorovich ranks for vectors
• Efficient computation for large-scale testing