Discrete Hamiltonian-Assisted Metropolis Sampling

2025-07-13

AI Summary

This work addresses efficient gradient-driven sampling from discrete distributions. We propose a novel Hamiltonian Monte Carlo (HMC) method tailored for discrete domains. Our key innovation is the first incorporation of continuous momentum variables and Hamiltonian dynamics into discrete sampling, achieved by constructing an augmented Hamiltonian distribution that satisfies generalized detailed balance; under a linear potential, this yields rejection-free, irreversible state transitions. The method integrates auxiliary-variable proposals, momentum reversal with gradient correction, and over-relaxation of state variables to yield a differentiable, fully discrete sampling procedure. Experiments demonstrate substantial improvements in sampling efficiency and convergence speed on binary and ordinal distribution tasks, alongside enhanced exploration of target distributions. Our approach establishes a new paradigm for discrete Bayesian inference.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Planning, Routing, and Scheduling: Mixed Discrete/Continuous Planning
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Gradient-based Markov Chain Monte Carlo methods have recently received much attention for sampling discrete distributions, with interesting connections to their continuous counterparts. For examples, there are two discrete analogues to the Metropolis-adjusted Langevin Algorithm (MALA). As motivated by Hamiltonian-Assisted Metropolis Sampling (HAMS), we propose Discrete HAMS (DHAMS), a discrete sampler which, for the first time, not only exploits gradient information but also incorporates a Gaussian momentum variable and samples a Hamiltonian as an augmented distribution. DHAMS is derived through several steps, including an auxiliary-variable proposal scheme, negation and gradient correction for the momentum variable, and over-relaxation for the state variable. Two distinctive properties are achieved simultaneously. One is generalized detailed balance, which enables irreversible exploration of the target distribution. The other is a rejection-free property for a target distribution with a linear potential function. In experiments involving both ordinal and binary distributions, DHAMS algorithms consistently yield superior performance compared with existing algorithms.

Problem

• Develops Discrete HAMS for sampling discrete distributions efficiently
• Incorporates gradient information and Gaussian momentum for enhanced sampling
• Achieves generalized detailed balance and rejection-free sampling properties

Innovation

• Discrete HAMS incorporates Gaussian momentum variable
• DHAMS uses gradient correction and over-relaxation
• DHAMS achieves generalized detailed balance irreversibly