MIRA: A Score for Conditional Distribution Accuracy and Model Comparison

2026-05-03

AI Summary

This work addresses the challenge of accurately evaluating and comparing candidate conditional distributions using only joint samples drawn from the true data-generating process. The authors propose the MIRA score, a novel evaluation metric grounded in the principle that the true and candidate conditional distributions should assign identical probability mass across all regions of the support. By constructing an analytic statistic that directly quantifies consistency between a candidate conditional distribution and the true generative mechanism, MIRA enables unbiased validation without requiring marginal likelihood computation. To the best of the authors’ knowledge, this is the first method to achieve such validation solely from joint samples, while also providing a theoretical reference value and uncertainty estimates. Empirical results on synthetic benchmarks and Bayesian inference tasks demonstrate that MIRA facilitates accurate assessment and reliable model comparison for conditional distributions.

Abstract

We introduce Mira, a sample-based score for assessing the accuracy of a candidate conditional distribution using only joint samples from the true data-generating process. Relying on the principle that distributions coincide if they assign equal probability mass to all regions, we derive an analytic expression for the Mira statistic, whose average defines the Mira score. This formulation further allows us to compute theoretical reference values and uncertainty estimates when the candidate distribution matches the true one. This framework enables model comparison by quantifying the alignment between the conditional distribution of a candidate model and the true data generating process. Consequently, Mira enables Bayesian model comparison through direct posterior validation, bypassing the challenging evidence computation. We demonstrate its effectiveness across several toy problems and Bayesian inference tasks.

Problem

• conditional distribution
• model comparison
• distribution accuracy
• Bayesian inference
• posterior validation

Innovation

• conditional distribution
• model comparison
• posterior validation
• score function
• Bayesian inference