The underlap coefficient as a measure of a biomarker's discriminatory ability

2025-04-16

AI Summary

Existing multiclass biomarker evaluation metrics—such as the VUS and trichotomous Youden index—rely on the stringent assumption of random ordering, which is frequently violated in the presence of confounding covariates or high-dimensional feature screening, leading to biased discrimination assessments. To address this, we propose the *underlap coefficient*, the first nonparametric multiclass discriminability measure that does not require the random-ordering assumption; it quantifies distributional overlap among ≥3 groups. We further develop a Bayesian nonparametric estimation framework enabling both unconditional and covariate-specific underlap estimation. Simulation studies demonstrate its robustness under model misspecification and complex dependency structures. Applied to Alzheimer’s disease data (cognitively normal, mild cognitive impairment, dementia), the method quantitatively compares the diagnostic efficacy of four biomarkers and uncovers age- and sex-specific modulation effects. This work establishes a theoretically more robust and practically more flexible paradigm for evaluating multiclass diagnostic biomarkers.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Graphical Models
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

The first step in evaluating a potential diagnostic biomarker is to examine the variation in its values across different disease groups. In a three-class disease setting, the volume under the receiver operating characteristic surface and the three-class Youden index are commonly used summary measures of a biomarker's discriminatory ability. However, these measures rely on a stochastic ordering assumption for the distributions of biomarker outcomes across the three groups. This assumption can be restrictive, particularly when covariates are involved, and its violation may lead to incorrect conclusions about a biomarker's ability to distinguish between the three disease classes. Even when a stochastic ordering exists, the order may vary across different biomarkers in discovery studies involving dozens or even thousands of candidate biomarkers, complicating automated ranking. To address these challenges and complement existing measures, we propose the underlap coefficient, a novel summary index of a biomarker's ability to distinguish between three (or more) disease groups, and study its properties. Additionally, we introduce Bayesian nonparametric estimators for both the unconditional underlap coefficient and its covariate-specific counterpart. These estimators are broadly applicable to a wide range of biomarkers and populations. A simulation study reveals a good performance of the proposed estimators across a range of conceivable scenarios. We illustrate the proposed approach through an application to an Alzheimer's disease (AD) dataset aimed to assess how four potential AD biomarkers distinguish between individuals with normal cognition, mild impairment, and dementia, and how and if age and gender impact this discriminatory ability.

Problem

• Measure biomarker's discriminatory ability without strict distribution assumptions
• Address challenges in ranking biomarkers with varying stochastic orders
• Assess covariate impact on biomarker performance in multi-class diseases

Innovation

• Proposes underlap coefficient for biomarker discrimination
• Introduces Bayesian nonparametric estimators for coefficients
• Applies approach to Alzheimer's disease biomarker assessment