Synthetic Priors

2026-02-27

AI Summary

This work addresses the absence of Bayesian priors in generalized linear models (GLMs) that align with expert intuition about response probabilities. Building on Good’s notion of imaginary observations, the authors construct a composite prior for GLM coefficients by placing Beta priors on conditional means at expert-specified design points. Coupled with Pólya–Gamma data augmentation, this formulation enables conjugate Gibbs sampling. The proposed framework unifies ridge regression, catalytic priors, and prediction-driven inference under a common bias–variance tradeoff mechanism, offering an interpretable and easily elicited prior specification strategy. In the Challenger O-ring dataset, posterior predictions exhibit greater robustness; in a Phase II dose-finding trial for atopic dermatitis, the approach yields 3–6% narrower 95% credible intervals and up to a 2-percentage-point increase in decision probabilities compared to non-informative priors.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Bayesian inference in generalized linear models requires a prior on the coefficient vector $β$. Practitioners naturally reason about response probabilities at specific covariate values, not about abstract log-odds parameters. We develop synthetic priors: informative Bayesian priors for GLMs grounded in Good's device of imaginary observations -- the principle that every conjugate prior is equivalent to a likelihood on pseudo-data from the same exponential family. The conditional means prior of Bedrick (1996) elicits independent Beta priors on the conditional mean response at $p$ expert-chosen design points; the induced prior on $β$ is a product of binomial likelihoods at synthetic data points. Combined with Pólya-Gamma data augmentation \citep{polson2013}, the posterior admits an exact conjugate Gibbs sampler -- no tuning, no Metropolis step -- by treating the augmented dataset as a standard logistic regression. We show that ridge regression and catalytic priors \citep{huang2020} are instances of Good's device, and identify prediction-powered inference \citep{angelopoulos2023ppi} as a structural analogue in the frequentist setting -- all three mediate a variance-bias tradeoff through a single informativeness parameter. We illustrate the approach on two benchmark problems: the Challenger O-ring data \citep{dalal1989}, where the BCJ prior provides a more moderate posterior predictive at the 31°F launch temperature; and a Phase~II atopic dermatitis dose-finding trial ($n = 300$), where the synthetic prior narrows 95\% credible intervals by 3-6\% and raises decision probabilities by up to 2 percentage points relative to a flat prior.

Problem

• Bayesian inference
• generalized linear models
• informative priors
• synthetic priors
• coefficient vector

Innovation

• synthetic priors
• imaginary observations
• generalized linear models
• Pólya-Gamma augmentation
• Bayesian elicitation