Approximate Bayesian Computation with Statistical Distances for Model Selection

2024-10-28

AI Summary

For model selection under intractable likelihoods, conventional approximate Bayesian computation (ABC) relying on hand-crafted summary statistics suffers from information loss and uncontrolled posterior approximation bias. This paper systematically develops and extends the full-data ABC framework: it directly compares observed and simulated datasets using statistical distances—including the Wasserstein distance and maximum mean discrepancy (MMD)—bypassing summary statistics and likelihood evaluation entirely. The framework integrates rejection sampling and sequential Monte Carlo for principled model comparison. Theoretically, it yields posterior approximations provably closer to the true posterior. Empirically, it achieves significantly higher model identification accuracy across multiple simulation studies and a real-world toad movement modeling task, demonstrating superior robustness, consistency, and generalizability over summary-statistic-based ABC methods.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Model selection is a key task in statistics, playing a critical role across various scientific disciplines. While no model can fully capture the complexities of a real-world data-generating process, identifying the model that best approximates it can provide valuable insights. Bayesian statistics offers a flexible framework for model selection by updating prior beliefs as new data becomes available, allowing for ongoing refinement of candidate models. This is typically achieved by calculating posterior probabilities, which quantify the support for each model given the observed data. However, in cases where likelihood functions are intractable, exact computation of these posterior probabilities becomes infeasible. Approximate Bayesian Computation (ABC) has emerged as a likelihood-free method and it is traditionally used with summary statistics to reduce data dimensionality, however this often results in information loss difficult to quantify, particularly in model selection contexts. Recent advancements propose the use of full data approaches based on statistical distances, offering a promising alternative that bypasses the need for summary statistics and potentially allows recovery of the exact posterior distribution. Despite these developments, full data ABC approaches have not yet been widely applied to model selection problems. This paper seeks to address this gap by investigating the performance of ABC with statistical distances in model selection. Through simulation studies and an application to toad movement models, this work explores whether full data approaches can overcome the limitations of summary statistic-based ABC for model choice.

Problem

• Develops ABC model selection using statistical distances
• Addresses intractable likelihoods in Bayesian model comparison
• Evaluates full-data approaches to overcome summary statistic limitations

Innovation

• ABC with statistical distances
• Full data approach bypasses summary statistics
• Simulation studies validate model selection performance