🤖 AI Summary
Biomedical machine learning is often compromised by data leakage arising from repeated measurements, study heterogeneity, batch effects, or temporal dependencies, leading to biased model evaluation. This work proposes a leakage-aware resampling workflow that innovatively integrates leakage-safe data partitioning, training-set-only preprocessing, nested hyperparameter tuning, and post-hoc leakage auditing, culminating in an interactive HTML diagnostic report. Built upon R’s S4 class system, the framework supports classification, regression, and survival analysis tasks while ensuring reproducibility and task-specific evaluation rigor. Simulation studies and multi-study transcriptomic case analyses demonstrate that leakage-preventive pipelines substantially alter model performance and downstream conclusions, underscoring their necessity and practical utility in robust biomedical machine learning.
📝 Abstract
Data leakage remains a recurrent source of optimistic bias in biomedical machine learning studies. Standard row-wise cross-validation and globally estimated preprocessing steps are often inappropriate for data with repeated measurements, study-level heterogeneity, batch effects, or temporal dependencies. This paper describes bioLeak, an R package for constructing leakage-aware resampling workflows and for auditing fitted models for common leakage mechanisms. The package provides leakage-aware split construction, train-fold-only preprocessing, cross-validated model fitting, nested hyperparameter tuning, post hoc leakage audits, and HTML reporting. The implementation supports binary classification, multiclass classification, regression, and survival analysis, with task-specific metrics and S4 containers for splits, fits, audits, and inflation summaries. The simulation artifacts show how apparent performance changes under controlled leakage mechanisms, and the case study illustrates how guarded and leaky pipelines can yield materially different conclusions on multi-study transcriptomic data. The emphasis throughout is on software design, reproducible workflows, and interpretation of diagnostic output.