Assessing Monotone Dependence: Area Under the Curve Meets Rank Correlation

2025-10-20

AI Summary

This paper addresses the lack of a unified framework for measuring monotonic dependence among random variables, where existing measures—such as Spearman’s rho and AUC—are applied disjointedly across continuous, binary, and ordinal discrete data. We propose two novel asymmetric dependence measures: Asymmetric Grade Correlation (AGC) and Coefficient of Monotonic Association (CMA), both defined via covariance-to-variance ratios under median distribution function transformations, thereby unifying Spearman correlation and ROC analysis within a common rank space. We establish theoretical guarantees, including consistency and asymptotic normality, and develop a DeLong-type statistical test for inference. The framework accommodates both continuous and discrete settings, with estimators possessing favorable large-sample properties. Empirical applications demonstrate its effectiveness in weather forecast calibration and uncertainty quantification for large language models.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Nonmonotonic Reasoning

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Web measurements

Abstract

The assessment of monotone dependence between random variables $X$ and $Y$ is a classical problem in statistics and a gamut of application domains. Consequently, researchers have sought measures of association that are invariant under strictly increasing transformations of the margins, with the extant literature being splintered. Rank correlation coefficients, such as Spearman's Rho and Kendall's Tau, have been studied at great length in the statistical literature, mostly under the assumption that $X$ and $Y$ are continuous. In the case of a dichotomous outcome $Y$, receiver operating characteristic analysis and the asymmetric area under the curve (AUC) measure are used to assess monotone dependence of $Y$ on a covariate $X$. Here we unify and extend thus far disconnected strands of literature, by developing common population level theory, estimators, and tests that bridge continuous and dichotomous settings and apply to all linearly ordered outcomes. In particular, we introduce asymmetric grade correlation, AGC$(X,Y)$, as the covariance of the mid distribution function transforms, or grades, of $X$ and $Y$, divided by the variance of the grade of $Y$. The coefficient of monotone association then is CMA$(X,Y) = frac{1}{2} ($AGC$(X,Y) + 1)$. When $X$ and $Y$ are continuous, AGC is symmetric and equals Spearman's Rho. When $Y$ is dichotomous, CMA equals AUC. We establish central limit theorems for the sample versions of AGC and CMA and develop a test of DeLong type for the equality of AGC or CMA values with a shared outcome $Y$. In case studies, we apply the new measures to assess progress in data-driven weather prediction, and to evaluate methods of uncertainty quantification for large language models.

Problem

• Unifying rank correlation and AUC measures for monotone dependence
• Bridging continuous and dichotomous outcome settings statistically
• Developing generalized association measures with population theory and tests

Innovation

• Introduces asymmetric grade correlation for monotone dependence
• Unifies rank correlation and AUC measures theoretically
• Develops statistical tests for new dependence measures