To select or not to select: predictively consistent priors instead of model selection

2026-06-22

AI Summary

Traditional Bayesian modeling relies on model selection to balance complexity and generalization, yet this approach often compromises predictive performance in small-sample settings. This work proposes a “predictive consistency prior” that maintains stability in the prior predictive distribution as model complexity increases, thereby circumventing explicit model selection. By shifting the modeling focus from parameter sparsity to constructing reasonable and stable priors in predictive space, the method reveals that the perceived necessity of model selection fundamentally stems from inadequate prior specification. The authors implement this prior in Bayesian linear and logistic regression, forward variable selection, and nonlinear models, demonstrating through numerical experiments that flexible models equipped with the predictive consistency prior match or even outperform carefully selected simpler models in out-of-sample prediction across a range of tasks.

Abstract

Bayesian modelling workflows often consider multiple candidate models of varying complexity. Model selection is commonly used to navigate potential trade-offs between model complexity and generalisability to new data. We study when model selection is unnecessary or can even be harmful for predictive performance in finite data regimes and find that the need for selecting simpler models can depend on prior choice. We formalise predictively consistent priors, which keep prior predictive implications stable as model complexity increases. Across examples and numerical experiments, including adding covariates in linear and logistic regression, forward variable selection, and nonlinear modelling, flexible models with predictively consistent priors typically match or outperform selected simpler models in out-of-sample predictive performance. When selection helps, it can indicate poor joint prior implications, such as excessive prior mass on implausible predictive values. Based on our findings, we propose replacing the notion of sparsity or parsimony at the level of model components with specifying priors that remain sensible in predictive space as models become more complex.

Problem

• model selection
• predictive performance
• Bayesian modeling
• prior choice
• model complexity

Innovation

• predictively consistent priors
• Bayesian model selection
• prior specification
• predictive performance
• model complexity