Parsimonious Subset Selection for Generalized Linear Models with Biomedical Applications

2026-03-23

AI Summary

This study addresses the challenge of optimal subset selection in generalized linear models (GLMs) for high-dimensional biomedical data, where computational intractability often compromises accuracy, sparsity, and interpretability. The authors propose COMBSS-GLM, a method that reformulates discrete subset selection as a continuous optimization problem via continuous Boolean relaxation. They develop an efficient Frank–Wolfe algorithm based on envelope gradients, which at each iteration fits only a single penalized GLM and traces a solution path across model sizes to yield sparse estimates. Theoretically, under specific curvature conditions, the relaxed objective is concave in the selection weights, guaranteeing that the global optimum lies at a binary vertex. Experiments demonstrate that COMBSS-GLM achieves more accurate variable selection and superior predictive performance than state-of-the-art penalized methods in logistic and multinomial regression tasks, successfully replicates known rice GWAS loci, and attains 100% test accuracy on the Khan SRBCT cancer dataset using only a small number of genes.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Optimization
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

High-dimensional biomedical studies require models that are simultaneously accurate, sparse, and interpretable, yet exact best subset selection for generalized linear models is computationally intractable. We develop a scalable method that combines a continuous Boolean relaxation of the subset problem with a Frank--Wolfe algorithm driven by envelope gradients. The resulting method, which we refer to as COMBSS-GLM, is simple to implement, requires one penalized generalized linear model fit per iteration, and produces sparse models along a model-size path. Theoretically, we identify a curvature-based parameter regime in which the relaxed objective is concave in the selection weights, implying that global minimizers occur at binary corners. Empirically, in logistic and multinomial simulations across low- and high-dimensional correlated settings, the proposed method consistently improves variable-selection quality relative to established penalised likelihood competitors while maintaining strong predictive performance. In biomedical applications, it recovers established loci in a binary-outcome rice genome-wide association study and achieves perfect multiclass test accuracy on the Khan SRBCT cancer dataset using a small subset of genes. Open-source implementations are available in R at https://github.com/benoit-liquet/COMBSS-GLM-R and in Python at https://github.com/saratmoka/COMBSS-GLM-Python.

Problem

• best subset selection
• generalized linear models
• high-dimensional data
• sparsity
• biomedical applications

Innovation

• best subset selection
• generalized linear models
• Frank-Wolfe algorithm
• Boolean relaxation
• sparse modeling