Estimating Marginal Likelihoods in Likelihood-Free Inference via Neural Density Estimation

2025-07-11

AI Summary

This work addresses the challenge of marginal likelihood (evidence) estimation in likelihood-free inference, where the likelihood function is intractable. We propose the first method to directly approximate the marginal likelihood from the output of sequential neural likelihood estimation (SNLE), without requiring additional simulations or posterior sampling. Leveraging only the density-ratio estimates obtained during SNLE training, our approach constructs an efficient and broadly applicable evidence estimator. Crucially, this is the first work to systematically integrate SNLE with Bayesian model comparison—bridging a key gap in simulation-based inference (SBI). It achieves improved estimation reliability over existing SBI model selection methods while preserving computational efficiency. Extensive experiments on multiple benchmark tasks demonstrate that our estimator closely approximates the true marginal likelihood. This advances both the theoretical foundations and practical applicability of neural density estimation–based inference for model selection.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Bayesian Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The marginal likelihood, or evidence, plays a central role in Bayesian model selection, yet remains notoriously challenging to compute in likelihood-free settings. While Simulation-Based Inference (SBI) techniques such as Sequential Neural Likelihood Estimation (SNLE) offer powerful tools to approximate posteriors using neural density estimators, they typically do not provide estimates of the evidence. In this technical report presented at BayesComp 2025, we present a simple and general methodology to estimate the marginal likelihood using the output of SNLE.

Problem

• Estimating marginal likelihoods in likelihood-free Bayesian inference
• Addressing computational challenges in evidence estimation via neural methods
• Extending SNLE to provide marginal likelihood approximations

Innovation

• Neural density estimation for likelihood-free inference
• Sequential Neural Likelihood Estimation (SNLE) technique
• Estimating marginal likelihoods in Bayesian model selection