Dependency-Aware Shrinkage Priors for High Dimensional Regression

2025-05-15

AI Summary

In high-dimensional regression, conventional shrinkage priors often ignore dependence structures among regression coefficients, leading to substantial estimation bias under strongly correlated predictors. This paper proposes a dependency-aware global-local shrinkage prior that explicitly incorporates predictor correlations into a continuous shrinkage framework—achieved by generalizing the Zellner g-prior to induce a structured coefficient covariance matrix. Theoretically, we characterize how dependency modeling affects prior and posterior distributions, and derive conditions under which it yields differential gains in coefficient recovery versus prediction. Monte Carlo simulations and empirical studies demonstrate that the method substantially improves coefficient estimation accuracy in strongly correlated settings, yet delivers only marginal improvements in predictive performance. Our findings underscore that dependency modeling must be carefully tailored to the inferential objective—e.g., interpretability versus prediction—thereby establishing a new paradigm for Bayesian sparse modeling in high dimensions.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Bayesian Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In high dimensional regression, global local shrinkage priors have gained significant traction for their ability to yield sparse estimates, improve parameter recovery, and support accurate predictive modeling. While recent work has explored increasingly flexible shrinkage prior structures, the role of explicitly modeling dependencies among coefficients remains largely unexplored. In this paper, we investigate whether incorporating such structures into traditional shrinkage priors improves their performance. We introduce dependency-aware shrinkage priors, an extension of continuous shrinkage priors that integrates correlation structures inspired by Zellner's g prior approach. We provide theoretical insights into how dependence alters the prior and posterior structure, and evaluate the method empirically through simulations and real data. We find that modeling dependence can improve parameter recovery when predictors are strongly correlated, but offers only modest gains in predictive accuracy. These findings suggest that prior dependence should be used selectively and guided by the specific inferential goals of the analysis.

Problem

• Exploring dependency modeling in high-dimensional regression shrinkage priors
• Investigating performance improvement by integrating coefficient dependencies
• Assessing impact of prior dependence on parameter recovery and prediction

Innovation

• Extends shrinkage priors with dependency modeling
• Integrates Zellner's g prior correlation structures
• Improves parameter recovery for correlated predictors