Structured Inference with Large Language Gibbs

2026-06-17

AI Summary

This work addresses the challenge of extracting structured knowledge from large language models (LLMs) in a probabilistically coherent manner to support complex variable reasoning. It proposes Large Language Gibbs (LLG), a method that embeds the LLM’s conditional distribution as a transition kernel within a Gibbs sampling framework. By iteratively resampling individual variables rather than generating sequences autoregressively, LLG avoids biases induced by sequential dependencies. This approach constitutes the first effective integration of LLMs with Markov chain Monte Carlo (MCMC), yielding a stationary distribution consistent with all local conditional distributions. Empirical results demonstrate that LLG excels in tasks including synthetic distribution sampling, consistency-aware reasoning, and Bayesian structure learning, establishing a new paradigm for structured probabilistic inference beyond single-pass generation.

Abstract

The knowledge encoded in large language models (LLMs) can serve as a substrate for structured reasoning over variables describing a complex world, but accessing this knowledge in a probabilistically coherent manner poses a difficult inference problem. We propose Large Language Gibbs, a scheme for structured probabilistic inference that uses conditional distributions of an LLM as transition operators. Rather than sampling structured objects through single-pass autoregressive generation, we iteratively resample individual variables conditioned on others using an LLM's next-token conditionals. This approach avoids order-dependent biases and produces a stationary distribution that reflects a compromise between all local conditionals. We apply this approach to sampling from synthetic distributions, consistent reasoning tasks, and Bayesian structure learning. The results suggest that the use of LLM conditionals in MCMC is a practical alternative to one-pass generation for structured probabilistic inference under a world prior accessible through noisy LLM conditionals.

Problem

• structured inference
• large language models
• probabilistic reasoning
• MCMC
• conditional distributions

Innovation

• Large Language Models
• Gibbs Sampling
• Structured Probabilistic Inference
• MCMC
• Conditional Distributions