Approximate Bayesian Computation sequential Monte Carlo via random forests

2024-06-22

AI Summary

In likelihood-intractable scenarios, Approximate Bayesian Computation (ABC) faces key bottlenecks: difficulty in selecting informative summary statistics, complex tuning of distance metrics and tolerance thresholds, high computational cost, and substantial posterior uncertainty due to weak prior information. To address these challenges, this paper proposes ABC-SMC-(D)RF—a novel method integrating Distributed Random Forests (DRF) into the Sequential Monte Carlo (SMC) framework for the first time. It enables end-to-end learning of discriminative features, eliminating manual design of summary statistics and distance functions. Coupled with adaptive tolerance scheduling and a pre-acceptance shrinkage mechanism, it iteratively concentrates sampling on high-probability parameter regions. Experiments across diverse deterministic and stochastic models demonstrate significant improvements in posterior estimation accuracy and robustness; notably, stability under weak priors is markedly enhanced, and computational efficiency surpasses that of conventional ABC-RF.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Probabilistic Programming
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Approximate Bayesian Computation (ABC) is a popular inference method when likelihoods are hard to come by. Practical bottlenecks of ABC applications include selecting statistics that summarize the data without losing too much information or introducing uncertainty, and choosing distance functions and tolerance thresholds that balance accuracy and computational efficiency. Recent studies have shown that ABC methods using random forest (RF) methodology perform well while circumventing many of ABC's drawbacks. However, RF construction is computationally expensive for large numbers of trees and model simulations, and there can be high uncertainty in the posterior if the prior distribution is uninformative. Here we adapt distributional random forests to the ABC setting, and introduce Approximate Bayesian Computation sequential Monte Carlo with random forests (ABC-SMC-(D)RF). This updates the prior distribution iteratively to focus on the most likely regions in the parameter space. We show that ABC-SMC-(D)RF can accurately infer posterior distributions for a wide range of deterministic and stochastic models in different scientific areas.

Problem

• Improves ABC inference via random forests for complex models
• Reduces computational cost and uncertainty in posterior estimation
• Enables accurate parameter inference across diverse scientific domains

Innovation

• Random forests for joint posterior inference
• Sequential Monte Carlo updating prior iteratively
• Distributional random forests direct parameter estimation