Adjusting for Heavy Censoring and Double-Dipping to Compare Risk Stratification Abilities of Existing Models for Time to Diagnosis of Huntington Disease

2025-11-05

AI Summary

Existing evaluations of Huntington’s disease (HD) diagnostic timing models—Langbehn, CAP, PIN, and MRS—are severely biased due to high right-censoring (>80%) and internal validation within training datasets. Method: We propose a censoring-corrected external validation framework, rigorously assessing risk stratification performance using survival analysis and robust metrics in the independent ENROLL-HD cohort. Contribution/Results: MRS achieves the highest discrimination; CAP and PIN follow closely while offering greater clinical feasibility. Sample size calculations in prior HD clinical trials—based on outdated models—systematically underestimate required enrollment, compromising statistical power. This study provides the first fair, external, and reproducible comparison of major HD prediction models under high-censoring conditions, establishing a methodological benchmark for precision prognostication and trial design.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Huntington disease (HD) is a genetically inherited neurodegenerative disease with progressively worsening symptoms. Accurately modeling time to HD diagnosis is essential for clinical trial design and treatment planning. Langbehn's model, the CAG-Age Product (CAP) model, the Prognostic Index Normed (PIN) model, and the Multivariate Risk Score (MRS) model have all been proposed for this task. However, differing in methodology, assumptions, and accuracy, these models may yield conflicting predictions. Few studies have systematically compared these models'performance, and those that have could be misleading due to (i) testing the models on the same data used to train them and (ii) failing to account for high rates of right censoring (80%+) in performance metrics. We discuss the theoretical foundations of the four most common models of time to HD diagnosis, offering intuitive comparisons about their practical feasibility. Further, we externally validate their risk stratification abilities using data from the ENROLL-HD study and performance metrics that adjust for censoring. Our findings guide the selection of a model for HD clinical trial design. The MRS model, which incorporates the most covariates, performed the best. However, the simpler CAP and PIN models were not far behind and may be logistically simpler to adopt. We also show how these models can be used to estimate sample sizes for an HD clinical trial, emphasizing that previous estimates would lead to underpowered trials.

Problem

• Systematically comparing four Huntington disease prediction models' conflicting performances
• Addressing biased evaluations from double-dipping and high censoring rates
• Validating models' risk stratification using censoring-adjusted metrics from ENROLL-HD data

Innovation

• External validation using ENROLL-HD study data
• Performance metrics adjusted for high censoring rates
• Comparing four models with sample size estimation