Prediction-based Inference in Electronic Health Record (EHR)-linked Biobanks

2026-03-15

AI Summary

This study addresses the challenge of heterogeneous missingness in biomarkers within electronic health records (EHRs), which—particularly under clinically driven observation mechanisms—can induce substantial bias in phenotypic analyses. The validity of prediction-based (PB) methods under such missingness remains unclear. We provide the first systematic classification and theoretical characterization of eight PB approaches alongside five conventional missing-data strategies. Through comprehensive evaluation across diverse EHR missingness mechanisms, integrating machine learning–enhanced PB, weighted complete-case analysis, and genome-wide association study (GWAS) frameworks, we assess statistical power, estimation accuracy, and Type I error control. Results demonstrate that, when model assumptions hold, PB methods substantially improve estimation efficiency and testing power, successfully replicating established genetic associations and enabling more powerful, population-representative biomarker GWAS in the All of Us cohort.

Technology Category

• Machine Learning: Ensemble Methods
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Reasoning under Uncertainty: Graphical Models

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Electronic health record (EHR)-linked biobank data facilitate large-scale scientific discoveries such as genome-wide association study (GWAS) on a multitude of phenotypic traits and biomarkers routinely captured in EHR. However, heterogeneous missingness in biomarkers may bias analyses when used as phenotypes. Recently proposed prediction-based (PB) inference methods incorporate external machine ML models to impute missing biomarkers and thereby improve the statistical power and estimation precision in association analyses. Yet, it remains unclear that if these methods are still preferable when the outcome $Y$ is generated under a clinically informative observation process. A comprehensive comparative evaluations and theoretical understanding of the existing PB methods under such realistic EHR missingness mechanisms are lacking. In this paper, we conduct a structured review of 8 recently developed PB methods and categorize them based on their frameworks. We systematically evaluate 9 methods, including 4 PB methods and 5 baseline methods from the traditional missing-data approaches, under 10 different outcome observation process models across different missing assumptions for continuous and binary outcomes. Our results show that PB methods can substantially improve statistical power and estimation efficiency when their missingness assumptions hold, but they may require stronger assumptions than CCA to control type I error. We provide theoretical results to characterize the scenarios under which CCA or PB methods may remain valid. Finally, we apply CCA, weighted CCA, and two PB methods to perform GWAS of six laboratory biomarkers in the All of Us data. The results demonstrate that PB methods can replicate known genetic associations while improving efficiency relative to (weighted) CCA. Furthermore, they extend inferential results to a more representative study population.

Problem

• Electronic Health Records
• missing data
• prediction-based inference
• biomarkers
• observation process

Innovation

• prediction-based inference
• EHR-linked biobanks
• missing data mechanisms
• genome-wide association study
• statistical efficiency