Bayesian Model Averaging under Predictor Redundancy via Density-Ratio Posterior Compression

2026-06-19

AI Summary

This work addresses the challenge that predictive variable redundancy causes Bayesian model averaging posteriors to spread thinly across numerous nearly equivalent support sets, hindering the generation of stable and interpretable summaries. To tackle this issue, the authors propose a density-ratio–based posterior compression framework that reports the posterior via hard or soft support regions without altering the original Bayesian objective. They introduce a computable density-ratio distortion measure and diagnostic tools, derive exact error expressions and validation bounds for region-based reporting, and establish theoretical conditions under which a small number of regions can effectively replace a large collection of individual support sets. Experiments demonstrate that the method yields more concise and interpretable summaries while preserving essential posterior information, and that the density-ratio diagnostics reliably detect cases of excessive information loss.

Abstract

Bayesian model averaging in support-indexed regression induces a posterior distribution over active predictor supports. Under predictor redundancy, posterior mass can spread across many nearly interchangeable supports, making exact-support summaries unstable or hard to interpret even when prediction is stable. We study how to report an already fitted Bayesian model averaging posterior without changing the Bayesian target. A report uses hard or soft regions of support space, and its compressed reporting law is compared with the reference posterior through an explicit density ratio. This ratio gives computable total-variation and Kullback--Leibler distortion, bounds for bounded predictive summaries, retained-mass diagnostics, and fallback-weight diagnostics. The framework covers fixed hard regions, metric-ball regions, posterior-cluster regions, and pooled-pruned region dictionaries. We prove exact error formulas and validation bounds for these region reports, and give conditions under which a few regions can replace a long list of individual supports. In simulations, our region reports often give shorter and clearer summaries while preserving the main posterior information, and the density-ratio diagnostics show when too much information has been lost.

Problem

• Bayesian model averaging
• predictor redundancy
• posterior compression
• support selection
• density ratio

Innovation

• Bayesian model averaging
• predictor redundancy
• density-ratio posterior compression
• support-space regions
• posterior summarization