Exploring the generalizability of the optimal 0.234 acceptance rate in random-walk Metropolis and parallel tempering algorithms

2024-08-13

AI Summary

The classical “0.234 optimal acceptance rate” rule for Random Walk Metropolis (RWM) and Parallel Tempering (PT) lacks rigorous characterization of its validity boundaries—particularly in low-dimensional, non-i.i.d., and multimodal target distributions. Method: We conduct theoretical analysis and large-scale numerical simulations to quantify the robustness and failure thresholds of this rule under anisotropic target densities and varying dimensionalities, and propose a novel adaptive temperature ladder strategy for PT anchored at the exchange acceptance rate of 0.234. Contribution/Results: We find that while the 0.234 rule remains approximately optimal in many low-dimensional settings, it degrades rapidly under strong anisotropy, revealing a dimension-dependent failure mechanism. Our adaptive ladder strategy—evaluated via Expected Squared Jump Distance (ESJD)—significantly improves sampling efficiency and convergence stability. This work provides both theoretical foundations and practical guidelines for deploying MCMC methods on complex, realistic target distributions.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

For random-walk Metropolis (RWM) and parallel tempering (PT) algorithms, an asymptotic acceptance rate of around 0.234 is known to be optimal in the high-dimensional limit. However, its practical relevance is uncertain due to restrictive derivation conditions. We synthesise previous theoretical advances in extending the 0.234 acceptance rate to more general settings, and demonstrate its applicability with a comprehensive empirical simulation study on examples examining how acceptance rates affect Expected Squared Jumping Distance (ESJD). Our experiments show the optimality of the 0.234 acceptance rate for RWM is surprisingly robust even in lower dimensions across various proposal and multimodal target distributions which may or may not have an i.i.d. product density. Parallel tempering experiments also show that the idealized 0.234 spacing of inverse temperatures may be approximately optimal for low dimensions and non i.i.d. product target densities, and that constructing an inverse temperature ladder with spacings given by a swap acceptance of 0.234 is a viable strategy. However, we observe the applicability of the 0.234 acceptance rate heuristic diminishes for both RWM and PT algorithms below a certain dimension which differs based on the target density, and that inhomogeneously scaled components in the target density further reduces its applicability in lower dimensions.

Problem

• Extending optimal 0.234 acceptance rate to general settings
• Testing robustness of 0.234 rate in non-i.i.d. distributions
• Validating 0.234 spacing for parallel tempering algorithms

Innovation

• Extends 0.234 acceptance rate to general settings
• Validates robustness in lower-dimensional multimodal distributions
• Uses 0.234 spacing for parallel tempering optimization