Sampling from Energy distributions with Target Concrete Score Identity
This work addresses the problem of efficient, unbiased sampling from unnormalized densities defined over discrete state spaces. We propose Time-Continuous Score-based Importance Sampling (TCSIS), a novel method grounded in the Target Concrete Score identity—a theoretical contribution that for the first time establishes an analytical link between marginal transition probabilities and ratios of Boltzmann-factor expectations, enabling self-normalized score estimation without target samples or the intractable partition function. TCSIS builds upon a forward uniform-noise continuous-time Markov chain (CTMC) and employs a neural network to model the Concrete Score, with expectations estimated via Monte Carlo. We design two variants: Self-Normalized TCSIS and Unbiased TCSIS. Empirical evaluation on statistical physics tasks demonstrates substantial improvements in both sampling efficiency and accuracy for discrete generative models, establishing a new paradigm for discrete sampling under unnormalized distributions.