High-dimensional regression with outcomes of mixed-type using the multivariate spike-and-slab LASSO

2025-06-16

AI Summary

This paper addresses joint modeling of mixed binary and continuous response variables in high-dimensional multivariate regression, where both the number of covariates $p$ and the number of responses $q$ diverge with the sample size $n$. We simultaneously estimate the sparse regression coefficient matrix $B$ and the residual precision matrix $Omega$. A latent-variable-based unified framework is proposed, which— for the first time—extends multivariate spike-and-slab priors to mixed-output settings. To enable efficient inference, we introduce a continuous relaxation of the prior and develop a Monte Carlo EM algorithm. Theoretically, we establish posterior contraction and sure screening properties under high-dimensional asymptotics. Extensive simulations and real-data applications in medicine and ecology demonstrate substantial improvements in prediction accuracy and variable selection consistency. Moreover, the method achieves asymptotic full identification of nonzero regression coefficients.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

We consider a high-dimensional multi-outcome regression in which $q,$ possibly dependent, binary and continuous outcomes are regressed onto $p$ covariates. We model the observed outcome vector as a partially observed latent realization from a multivariate linear regression model. Our goal is to estimate simultaneously a sparse matrix ($B$) of latent regression coefficients (i.e., partial covariate effects) and a sparse latent residual precision matrix ($Omega$), which induces partial correlations between the observed outcomes. To this end, we specify continuous spike-and-slab priors on all entries of $B$ and off-diagonal elements of $Omega$ and introduce a Monte Carlo Expectation-Conditional Maximization algorithm to compute the maximum a posterior estimate of the model parameters. Under a set of mild assumptions, we derive the posterior contraction rate for our model in the high-dimensional regimes where both $p$ and $q$ diverge with the sample size $n$ and establish a sure screening property, which implies that, as $n$ increases, we can recover all truly non-zero elements of $B$ with probability tending to one. We demonstrate the excellent finite-sample properties of our proposed method, which we call mixed-mSSL, using extensive simulation studies and three applications spanning medicine to ecology.

Problem

• Estimates sparse regression coefficients for mixed-type outcomes
• Models dependencies among outcomes via sparse precision matrix
• Develops scalable algorithm for high-dimensional data analysis

Innovation

• Multivariate spike-and-slab LASSO for regression
• Monte Carlo Expectation-Conditional Maximization algorithm
• Sparse latent residual precision matrix estimation