Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective

2025-06-06

AI Summary

To address distribution distortion, reduced diversity, and low efficiency in language model sampling under hard constraints, this paper proposes the first Metropolis-Hastings (MH)-based MCMC constrained sampling framework. The method constructs a proposal distribution guided by the language model’s likelihood, ensuring strict adherence to hard constraints while provably converging to the true conditional distribution. Compared with existing constrained decoding approaches, our framework significantly improves sample quality, syntactic and semantic validity, and diversity—demonstrated on both synthetic benchmarks and real-world program fuzz testing tasks—while retaining high generation efficiency. The core contribution lies in the systematic integration of the MH mechanism into language model constrained sampling for the first time, thereby unifying constraint satisfaction, theoretical correctness, and computational feasibility.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Constraint Programming
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Constrained decoding enables Language Models (LMs) to produce samples that provably satisfy hard constraints. However, existing constrained-decoding approaches often distort the underlying model distribution, a limitation that is especially problematic in applications like program fuzzing, where one wants to generate diverse and valid program inputs for testing purposes. We propose a new constrained sampling framework based on Markov Chain Monte Carlo (MCMC) that simultaneously satisfies three core desiderata: constraint satisfying (every sample satisfies the constraint), monotonically converging (the sampling process converges to the true conditional distribution), and efficient (high-quality samples emerge in few steps). Our method constructs a proposal distribution over valid outputs and applies a Metropolis-Hastings acceptance criterion based on the LM's likelihood, ensuring principled and efficient exploration of the constrained space. Empirically, our sampler outperforms existing methods on both synthetic benchmarks and real-world program fuzzing tasks.

Problem

• Ensures LM samples satisfy hard constraints
• Maintains model distribution without distortion
• Efficiently generates diverse valid outputs

Innovation

• MCMC-based constrained sampling framework
• Metropolis-Hastings acceptance criterion
• Efficient exploration of constrained space