Machine learning models for estimating counterfactuals in a single-arm inflammatory bowel disease study

2026-04-25

AI Summary

This study addresses the challenge of evaluating adalimumab (ADA) efficacy in pediatric Crohn’s disease within single-arm trials lacking a control group. To overcome this limitation, the authors systematically constructed a virtual comparator by leveraging external infliximab (IFX) treatment data and training five machine learning models—including LightGBM—to predict counterfactual outcomes for ADA-treated patients at one year, specifically steroid-free clinical remission and composite remission. The predicted treatment effects were validated against results from propensity score matching. Among the models, LightGBM yielded estimates closest to the matching benchmark, and no statistically significant differences were observed between ADA and IFX in either primary or secondary endpoints. These findings demonstrate the feasibility and validity of machine learning–driven virtual controls as a robust alternative to conventional control groups in pediatric inflammatory bowel disease research.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Game Theory and Economic Paradigms: Adversarial Learning
• Multiagent Systems: Adversarial Agents

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Single-arm trials accelerate study timelines by reducing the number of patients that must be recruited for a concurrent control group. However, these designs require an alternative comparator to estimate treatment effects. One approach is to construct a virtual control arm using a machine learning (ML) model trained on external control data to predict the counterfactual outcomes of the treatment arm. Our aim in this study was to leverage virtual controls by developing and evaluating ML-based counterfactual outcome models trained on IFX-treated patients to predict 1-year steroid-free clinical remission (SFCR ) and a composite of C-reactive protein remission plus steroid-free clinical remission (CRP-SFCR) for ADA-treated pediatric Crohn's disease patients, and to compare the resulting IFX-versus-ADA treatment effect estimates with those obtained using propensity score matching to external controls. Five ML models were used to train counterfactual models on the observed IFX cohort data. The resulting models were used to predict the counterfactual outcomes for the ADA arm patients. LGBM yields the best OR closest to the propensity score matched reference, and all 95% CI results align with the conclusion from the reference study that no statistical difference in the primary and secondary outcomes has been observed between the patients treated with ADA or IFX. Our study supports virtual controls as a viable and effective substitute for expensive, lengthy or unethical patient recruitment in an inflammatory bowel disease (IBD) trial. The developed gradient boosted prediction model can be used as a pretrained model to generate IFX counterfactual predictions in future studies, pending external validation and assessment of transportability.

Problem

• counterfactual estimation
• single-arm trial
• inflammatory bowel disease
• virtual control arm
• treatment effect

Innovation

• virtual control arm
• counterfactual prediction
• machine learning
• single-arm trial
• gradient boosting