A note on conditional densities, Bayes' rule, and recent criticisms of Bayesian inference

2026-03-27

AI Summary

This study addresses recent preprint critiques questioning the consistency of Bayesian inference by clarifying the mathematical foundations of conditional densities. It demonstrates that practical Bayesian updating does not condition directly on zero-probability events; rather, it first conditions on a random variable and subsequently substitutes the observed value. By rigorously distinguishing formal conditioning from the actual inferential procedure—and integrating measure-theoretic formalism with intuitive interpretation—the work systematically exposes mathematical misapplications in the critical literature. The analysis confirms that, when correctly understood and implemented, Bayesian inference remains logically coherent and free of internal contradictions, thereby reinforcing its theoretical validity and dispelling prevalent misconceptions.

Abstract

When performing Bayesian inference, we frequently need to work with conditional probability densities. For example, the posterior function is the conditional density of the parameters given the data. Some might worry that conditional densities are ill-defined, considering that for a continuous random variable $Y$, the event $\{Y=y\}$ has probability zero, meaning the formula $\mathbb{P}(A|B)=\mathbb{P}(A\cap B)/\mathbb{P}(B)$ is inapplicable. In reality, when we work with conditional densities, we never condition directly on the zero-probability event $\{Y=y\}$; rather, we first condition on the random variable $Y$, and then we may plug in an observed value $y$. The first purpose of our article is to provide an exposition on conditional densities that elaborates on this point. While we have aimed to make this explanation accessible, we follow it with a roadmap of the measure theory needed to make it rigorous. A recent preprint (arXiv:2411.13570) has expressed the concern that probability densities are ill-defined and that as a result Bayes' theorem cannot be used, and they provide examples that allegedly demonstrate inconsistencies in the Bayesian framework. The second purpose of our article is to investigate their claims. We contend that the examples given in their work do not demonstrate any inconsistencies; we find that there are mathematical errors and that they deviate significantly from the Bayesian framework.

Problem

• conditional densities
• Bayesian inference
• Bayes' theorem
• probability zero events
• foundations of probability

Innovation

• conditional densities
• Bayes' theorem
• measure theory
• Bayesian inference
• zero-probability events