Heteroscedastic Double Bayesian Elastic Net

2025-02-04

AI Summary

Addressing the dual challenges of heteroscedastic errors and ultra-high dimensionality (where the number of predictors far exceeds the sample size) in regression, this paper proposes a hierarchical Bayesian method that jointly models the conditional mean and log-variance. It is the first to embed simultaneous sparse selection and grouped-effect modeling for both mean and variance parameters within a two-level Bayesian elastic net framework. The approach employs a fused ℓ₁/ℓ₂-penalized prior, implements posterior inference via Gibbs sampling, and rigorously establishes posterior concentration, variable selection consistency, and asymptotic normality. Simulation studies demonstrate that the method significantly outperforms existing approaches in estimation accuracy, robustness of variable selection, and identification of complex heteroscedastic structures. This work thus provides a novel paradigm for high-dimensional heteroscedastic regression, uniquely combining theoretical guarantees with practical efficacy.

Technology Category

• Machine Learning: Ensemble Methods
• Reasoning under Uncertainty: Relational Probabilistic Models
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

In many practical applications, regression models are employed to uncover relationships between predictors and a response variable, yet the common assumption of constant error variance is frequently violated. This issue is further compounded in high-dimensional settings where the number of predictors exceeds the sample size, necessitating regularization for effective estimation and variable selection. To address this problem, we propose the Heteroscedastic Double Bayesian Elastic Net (HDBEN), a novel framework that jointly models the mean and log-variance using hierarchical Bayesian priors incorporating both $ell_1$ and $ell_2$ penalties. Our approach simultaneously induces sparsity and grouping in the regression coefficients and variance parameters, capturing complex variance structures in the data. Theoretical results demonstrate that proposed HDBEN achieves posterior concentration, variable selection consistency, and asymptotic normality under mild conditions which justifying its behavior. Simulation studies further illustrate that HDBEN outperforms existing methods, particularly in scenarios characterized by heteroscedasticity and high dimensionality.

Problem

• Addresses heteroscedasticity in regression models
• Handles high-dimensional predictor spaces
• Combines sparsity and grouping in coefficients

Innovation

• Heteroscedastic Double Bayesian Elastic Net
• Hierarchical Bayesian priors
• Sparsity and grouping effects