Variational approximate penalized credible regions for Bayesian grouped regression

2026-01-23

AI Summary

This study addresses the high computational cost and ambiguous variable selection commonly encountered in Bayesian high-dimensional grouped regression. The authors propose a fast grouped penalized credible region method based on group-wise global-local shrinkage priors and variational inference. By reformulating the penalized credible region as a convex optimization problem, they establish—for the first time in high-dimensional settings—theoretical guarantees of both parameter estimation consistency and variable selection consistency for the resulting estimator, along with convergence guarantees for the coordinate ascent algorithm. The method demonstrates superior performance over existing approaches in scenarios such as ANOVA and nonparametric varying-coefficient models, achieving state-of-the-art results in variable selection accuracy, predictive precision, and computational efficiency.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We develop a fast and accurate grouped penalized credible region approach for variable selection and prediction in Bayesian high-dimensional linear regression. Most existing Bayesian methods either are subject to high computational costs due to long Markov Chain Monte Carlo runs or yield ambiguous variable selection results due to non-sparse solution output. The penalized credible region framework yields sparse post-processed estimates that facilitates unambiguous grouped variable selection. High estimation accuracy is achieved by shrinking noise from unimportant groups using a grouped global-local shrinkage prior. To ensure computational scalability, we approximate posterior summaries using coordinate ascent variational inference and recast the penalized credible region framework as a convex optimization problem that admits efficient computations. We prove that the resultant post-processed estimators are both parameter-consistent and variable selection consistent in high-dimensional settings. Theory is developed to justify running the coordinate ascent algorithm for at least two cycles. Through extensive simulations, we demonstrate that our proposed method outperforms state-of-the-art methods in grouped variable selection, prediction, and computation time for several common models including ANOVA and nonparametric varying coefficient models.

Problem

• Bayesian grouped regression
• variable selection
• high-dimensional regression
• penalized credible regions
• sparse estimation

Innovation

• penalized credible region
• grouped variable selection
• variational inference
• global-local shrinkage prior
• Bayesian high-dimensional regression