Score-Repellent Monte Carlo: Toward Efficient Non-Markovian Sampler with Constant Memory in General State Spaces

2026-04-24

AI Summary

This work addresses the inefficiency of existing history-based Monte Carlo methods in high-dimensional state spaces, where encoding trajectory history often leads to redundant sampling and high variance. The authors propose the Score-Repellent Monte Carlo (SRMC) framework, which compresses historical information by maintaining a running average of the score function in ℝᵈ and introduces an exponentially tilted score to construct a dynamic, unnormalized surrogate target, enabling non-Markovian sampling. SRMC generalizes near-zero-variance history-dependent sampling—previously limited to finite state spaces—to general state spaces, requiring only O(d) constant memory. The asymptotic variance decreases as O(1/α) with increasing repellent strength α. Combining stochastic approximation, controlled Markov noise analysis, and generic MCMC kernels, the method significantly reduces estimation variance and improves mode coverage on both continuous distributions and discrete energy-based models, while preserving low memory overhead and computational efficiency.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

History-dependent sampling can reduce long-run Monte Carlo variance by discouraging redundant revisits, but existing schemes typically encode history through empirical measure on finite state spaces, which is infeasible in high-dimensional discrete configuration spaces or ill-posed in continuous domains. We propose Score-Repellent Monte Carlo (SRMC) framework that summarizes trajectory history by a running average of score evaluations in $R^d$, where $d$ is the dimension of the score and state representation. This history is converted into a surrogate target through an exponential score tilt, indexed with $α$ that represents the strength of repellence in controlling the magnitude of the history-based repulsion. The surrogate family is normalization-free in the standard MCMC sense, yielding a generic wrapper: at each iteration, any base kernel targeting $π$ can instead be run on the current surrogate $π_{θ_n}$ while the history is updated online. We analyze the coupled evolution of the history recursion and Monte Carlo estimators using stochastic approximation with controlled Markovian noise, establishing almost sure convergence and a joint central limit theorem. We further identify regimes in which the asymptotic covariance decreases as $α$ increases, with scaling $O(1/α)$, extending the near-zero-variance effect of finite-state history-dependent samplers to general state spaces with constant memory. Experiments on continuous targets and discrete energy-based models demonstrate improved estimator variance and mode coverage, while retaining $O(d)$ memory usage and modest per-iteration overhead.

Problem

• non-Markovian sampling
• history-dependent Monte Carlo
• general state spaces
• variance reduction
• constant memory

Innovation

• Score-Repellent Monte Carlo
• non-Markovian sampling
• constant memory
• score-based repulsion
• stochastic approximation