Polynomial Log-Marginals and Tweedie's Formula : When Is Bayes Possible?

2025-09-06

AI Summary

This paper investigates the theoretical feasibility of directly modeling the logarithm of the marginal density in empirical Bayes: specifically, which polynomial forms of the log-marginal density correspond to valid priors? Leveraging the Tweedie formula, a compound decision framework, analytic continuation via the heat equation, and convexity analysis, we rigorously characterize the relationship between marginal densities and prior realizability. We establish that log-marginal densities expressible as polynomials of degree three or higher cannot arise from any proper prior—thereby excluding all exponential-family priors—and that only quadratic log-marginals uniquely correspond to Gaussian priors. This work provides the first precise characterization of the Bayesian realizability boundary for polynomial log-marginals, revealing a fundamental reason why many widely used empirical Bayes methods lack formal Bayesian justification. Moreover, it strengthens analytical diagnostic tools for identifying Gaussian convolutions.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Motivated by Tweedie's formula for the Compound Decision problem, we examine the theoretical foundations of empirical Bayes estimators that directly model the marginal density $m(y)$. Our main result shows that polynomial log-marginals of degree $k ge 3 $ cannot arise from any valid prior distribution in exponential family models, while quadratic forms correspond exactly to Gaussian priors. This provides theoretical justification for why certain empirical Bayes decision rules, while practically useful, do not correspond to any formal Bayes procedures. We also strengthen the diagnostic by showing that a marginal is a Gaussian convolution only if it extends to a bounded solution of the heat equation in a neighborhood of the smoothing parameter, beyond the convexity of $c(y)= frac12 y^2+log m(y)$.

Problem

• Identifies invalid prior distributions for polynomial log-marginals
• Theoretically justifies empirical Bayes rules lacking formal Bayes correspondence
• Strengthens diagnostic for Gaussian convolution in marginal distributions

Innovation

• Polynomial log-marginals invalid for exponential families
• Quadratic log-marginals correspond to Gaussian priors
• Heat equation tests Gaussian convolution necessity