Easily Computed Marginal Likelihoods for Multivariate Mixture Models Using the THAMES Estimator

2025-04-30

AI Summary

This paper addresses the challenge of efficiently and accurately estimating the marginal likelihood for multivariate mixture models. We propose THAMES, a novel estimator based on MCMC samples that combines truncated harmonic means with an asymptotically optimal parameter ordering scheme. THAMES is the first method to deliver finite-variance, consistent, and asymptotically normal marginal likelihood estimates for arbitrary multivariate mixture models with any number of components—without requiring latent-variable sampling and inherently robust to label switching. Its ordering strategy ensures both computational efficiency and interpretability in parameter space. Extensive simulations and real-data analyses demonstrate that THAMES substantially outperforms existing approaches in accuracy, stability, and computational efficiency, maintaining superior performance even in high-dimensional and multi-component settings. Thus, THAMES provides a reliable tool for Bayesian model selection and inference.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We present a new version of the truncated harmonic mean estimator (THAMES) for univariate or multivariate mixture models. The estimator computes the marginal likelihood from Markov chain Monte Carlo (MCMC) samples, is consistent, asymptotically normal and of finite variance. In addition, it is invariant to label switching, does not require posterior samples from hidden allocation vectors, and is easily approximated, even for an arbitrarily high number of components. Its computational efficiency is based on an asymptotically optimal ordering of the parameter space, which can in turn be used to provide useful visualisations. We test it in simulation settings where the true marginal likelihood is available analytically. It performs well against state-of-the-art competitors, even in multivariate settings with a high number of components. We demonstrate its utility for inference and model selection on univariate and multivariate data sets.

Problem

• Estimating marginal likelihoods for multivariate mixture models
• Improving computational efficiency of MCMC-based estimators
• Enabling model selection without hidden allocation samples

Innovation

• THAMES estimator for multivariate mixture models
• Computes marginal likelihood from MCMC samples
• Invariant to label switching, no hidden samples needed