Variable Fusion and Selection via a Spike-and-Slab Approach with Nonlocal Priors

2026-04-28

AI Summary

This study addresses the challenge of simultaneously achieving variable fusion and selection in regression modeling by proposing a novel approach within the Bayesian model averaging framework. The method introduces latent variables to construct a discrete model space, thereby unifying the clustering of covariates—assigning them identical coefficients—and the elimination of irrelevant predictors. It innovatively employs a non-local prior tailored for fusion tasks as the slab component within a spike-and-slab architecture, coupled with an efficient Gibbs sampling scheme. Both theoretical analysis and empirical experiments demonstrate that the proposed method achieves superior performance in terms of fusion accuracy, variable selection consistency, and computational efficiency.

Abstract

Variable fusion in linear regression models is a statistical method that identifies covariates making similar contributions to the response variable and imposes the same coefficient values on them. Many methods for variable fusion also incorporate variable selection for practical reasons. In this paper, within the Bayesian model averaging (BMA) framework, we propose a spike-and-slab-based Bayesian method that performs both variable fusion and selection. This is challenging in the BMA framework because one must construct a discrete model space that accommodates both selection and fusion and assign suitable priors over that space. In the proposed method, we present a way to explore a model space for variable fusion and selection based on Gibbs sampling by devising a prior distribution for latent variables representing the model. Furthermore, among non-local priors with superior model selection properties, we construct a prior tailored for variable fusion and use it as the slab distribution. We examine the effectiveness of the proposed method through theoretical and empirical studies.

Problem

• variable fusion
• variable selection
• spike-and-slab
• nonlocal priors
• Bayesian model averaging

Innovation

• spike-and-slab prior
• variable fusion
• Bayesian model averaging
• non-local priors
• Gibbs sampling