ROOFS: RObust biOmarker Feature Selection

2026-01-08

AI Summary

This study addresses the instability of feature selection in biomedical data caused by high dimensionality, small sample sizes, multicollinearity, and missing values. To this end, we developed ROOFS, a Python toolkit that, for the first time, integrates multidimensional evaluation metrics—including realistic performance estimation based on semi-synthetic data—to systematically benchmark diverse feature selection methods. Our framework comprehensively assesses downstream predictive performance, stability, and feature reliability through variance inflation factor-based pre-screening, Benjamini–Hochberg–corrected joint statistical testing, optimism-corrected performance evaluation, and semi-synthetic data generation. Applied to the PIONeeR lung cancer immunotherapy dataset, ROOFS identified an optimal method among 253 models, significantly outperforming mainstream approaches such as LASSO and thereby enhancing the robustness and clinical translatability of biomarker discovery.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Search and Optimization: Metareasoning and Metaheuristics
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Feature selection (FS) is essential for biomarker discovery and in the analysis of biomedical datasets. However, challenges such as high-dimensional feature space, low sample size, multicollinearity, and missing values make FS non-trivial. Moreover, FS performances vary across datasets and predictive tasks. We propose roofs, a Python package available at https://gitlab.inria.fr/compo/roofs, designed to help researchers in the choice of FS method adapted to their problem. Roofs benchmarks multiple FS methods on the user's data and generates reports that summarize a comprehensive set of evaluation metrics, including downstream predictive performance estimated using optimism correction, stability, reliability of individual features, and true positive and false positive rates assessed on semi-synthetic data with a simulated outcome. We demonstrate the utility of roofs on data from the PIONeeR clinical trial, aimed at identifying predictors of resistance to anti-PD-(L)1 immunotherapy in lung cancer. The PIONeeR dataset contained 374 multi-source blood and tumor biomarkers from 435 patients. A reduced subset of 214 features was obtained through iterative variance inflation factor pre-filtering. Of the 34 FS methods gathered in roofs, we evaluated 23 in combination with 11 classifiers (253 models in total) and identified a filter based on the union of Benjamini-Hochberg false discovery rate-adjusted p-values from t-test and logistic regression as the optimal approach, outperforming other methods including the widely used LASSO. We conclude that comprehensive benchmarking with roofs has the potential to improve the robustness and reproducibility of FS discoveries and increase the translational value of clinical models.

Problem

• feature selection
• biomarker discovery
• high-dimensional data
• multicollinearity
• reproducibility

Innovation

• feature selection
• biomarker discovery
• benchmarking framework
• optimism correction
• stability evaluation