Refined Gradient-Based Temperature Optimization for the Replica-Exchange Monte-Carlo Method

2026-01-20

AI Summary

Replica Exchange Monte Carlo (REMC) suffers from limited sampling efficiency in multimodal distributions due to suboptimal temperature schedules. This work proposes a gradient-based online temperature optimization method that minimizes the variance of swap acceptance rates between adjacent replicas while explicitly enforcing physical constraints—such as monotonicity of inverse temperatures—through a reparameterization mechanism. This approach ensures stable optimization without requiring intricate hyperparameter tuning. Notably, it is the first to embed physical constraints directly within a gradient-based optimization framework. Experiments on the two-dimensional Ising and XY models, as well as the three-dimensional Edwards–Anderson spin glass model, demonstrate more uniform acceptance rates and significantly reduced round-trip times compared to existing policy gradient methods, highlighting its superior performance.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

The replica-exchange Monte-Carlo (RXMC) method is a powerful Markov-chain Monte-Carlo algorithm for sampling from multi-modal distributions, which are challenging for conventional methods. The sampling efficiency of the RXMC method depends highly on the selection of the temperatures, and finding optimal temperatures remains a challenge. In this study, we propose a refined online temperature selection method by extending the gradient-based optimization framework proposed previously. Building upon the existing temperature update approach, we introduce a reparameterization technique to strictly enforce physical constraints, such as the monotonic ordering of inverse temperatures, which were not explicitly addressed in the original formulation. The proposed method defines the variance of acceptance rates between adjacent replicas as a loss function, estimates its gradient using differential information from the sampling process, and optimizes the temperatures via gradient descent. We demonstrate the effectiveness of our method through experiments on benchmark spin systems, including the two-dimensional ferromagnetic Ising model, the two-dimensional ferromagnetic XY model, and the three-dimensional Edwards-Anderson model. Our results show that the method successfully achieves uniform acceptance rates and reduces round-trip times across the temperature space. Furthermore, our proposed method offers a significant advantage over recently proposed policy gradient method that require careful hyperparameter tuning, while simultaneously preventing the constraint violations that destabilize optimization.

Problem

• replica-exchange Monte Carlo
• temperature optimization
• multi-modal sampling
• acceptance rate
• physical constraints

Innovation

• replica-exchange Monte Carlo
• gradient-based optimization
• temperature scheduling
• reparameterization
• acceptance rate variance