ROC Analysis with Covariate Adjustment Using Neural Network Models: Evaluating the Role of Age in the Physical Activity-Mortality Association

2025-10-16

AI Summary

This study addresses the challenge of evaluating biomarker discriminative performance under confounding covariates (e.g., age, sex, BMI). We propose the first neural-network-based framework for covariate-adjusted ROC analysis, enabling flexible, nonlinear modeling of covariate–biomarker distribution relationships and yielding individualized AUC estimates—thereby overcoming restrictive linear or parametric assumptions. Validated on both simulated data and real-world data (predicting all-cause mortality from physical activity), our method reveals that the functional biomarker TAC exhibits significant predictive capacity, with its discriminative ability declining with age and being significantly modulated by sex and BMI. To our knowledge, this is the first work to integrate deep learning into covariate-adjusted ROC methodology, substantially improving accuracy and clinical interpretability in high-dimensional, nonlinear settings.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Graphical Models
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

The receiver operating characteristic (ROC) curve and its summary measure, the Area Under the Curve (AUC), are well-established tools for evaluating the efficacy of biomarkers in biomedical studies. Compared to the traditional ROC curve, the covariate-adjusted ROC curve allows for individual evaluation of the biomarker. However, the use of machine learning models has rarely been explored in this context, despite their potential to develop more powerful and sophisticated approaches for biomarker evaluation. The goal of this paper is to propose a framework for neural network-based covariate-adjusted ROC modeling that allows flexible and nonlinear evaluation of the effectiveness of a biomarker to discriminate between two reference populations. The finite-sample performance of our method is investigated through extensive simulation tests under varying dependency structures between biomarkers, covariates, and referenced populations. The methodology is further illustrated in a clinically case study that assesses daily physical activity - measured as total activity time (TAC), a proxy for daily step count-as a biomarker to predict mortality at three, five and eight years. Analyzes stratified by sex and adjusted for age and BMI reveal distinct covariate effects on mortality outcomes. These results underscore the importance of covariate-adjusted modeling in biomarker evaluation and highlight TAC's potential as a functional capacity biomarker based on specific individual characteristics.

Problem

• Develop neural network models for covariate-adjusted ROC analysis
• Evaluate biomarker discrimination with flexible nonlinear covariate adjustments
• Assess physical activity's mortality prediction adjusting for age effects

Innovation

• Neural network models for covariate-adjusted ROC analysis
• Flexible nonlinear biomarker evaluation framework
• Simulation tests for finite-sample performance validation