Time-dependent Predictive Accuracy Metrics in the Context of Interval Censoring and Competing Risks

2025-01-02

AI Summary

Evaluating time-varying prediction models under co-occurring interval censoring and competing risks remains challenging due to dynamic performance assessment over clinically relevant time windows. Method: We propose two complementary, model-driven evaluation strategies—model-based and inverse probability of censoring weighting (IPCW)—and systematically extend the AUC, Brier score, and expected prediction cross-entropy (EPCE) to dynamic assessment within a user-specified time window ([t, t+Delta t)). Contribution/Results: We introduce the first dual-path framework that jointly ensures full-risk-set modeling and precise event-window delineation. Simulation studies demonstrate that both methods achieve low bias, high stability, and strong clinical interpretability; together, they enable accurate, unbiased dynamic performance evaluation within ([t, t+Delta t)), substantially outperforming conventional static metrics. This work establishes a novel paradigm for dynamic validation of clinical risk prediction models under complex survival data structures.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Evaluating the performance of a prediction model is a common task in medical statistics. Standard accuracy metrics require the observation of the true outcomes. This is typically not possible in the setting with time-to-event outcomes due to censoring. Interval censoring, the presence of time-varying covariates, and competing risks present additional challenges in obtaining those accuracy metrics. In this study, we propose two methods to deal with interval censoring in a time-varying competing risk setting: a model-based approach and the inverse probability of censoring weighting (IPCW) approach, focusing on three key time-dependent metrics: area under the receiver-operating characteristic curve (AUC), Brier score, and expected predictive cross-entropy (EPCE). The evaluation is conducted over a medically relevant time interval of interest, $[t, Delta t)$. The model-based approach includes all subjects in the risk set, using their predicted risks to contribute to the accuracy metrics. In contrast, the IPCW approach only considers the subset of subjects who are known to be event-free or experience the event within the interval of interest. we performed a simulation study to compare the performance of the two approaches with regard to the three metrics.

Problem

• Addressing interval censoring in time-varying competing risk settings
• Proposing methods to evaluate time-dependent predictive accuracy metrics
• Comparing model-based and IPCW approaches for medical prediction models

Innovation

• Model-based approach for interval censoring
• IPCW method for competing risks
• Time-dependent AUC, Brier, EPCE metrics