Reasoning with Sampling: Cutting at Decision Points

2026-05-28

AI Summary

This work addresses the inefficiency of existing sampling-based inference methods, which struggle to effectively explore critical decision points during resampling. The authors propose a training-free inference optimization approach that identifies significant entropy spikes in the base model’s next-token prediction along the reasoning trajectory as proxies for key decision points. By integrating an entropy-guided cutoff selection strategy with the Metropolis-Hastings sampling algorithm, the method enables targeted resampling that substantially reduces mixing time complexity. Evaluated on multiple challenging benchmarks—including MATH500, HumanEval, GPQA Diamond, and AIME26—the approach consistently outperforms current baselines and even reinforcement learning–trained models, demonstrating strong training-free reasoning capabilities.

Abstract

Frontier reasoning models are produced by posttraining base language models with reinforcement learning. Recent work has challenged this by showing that sampling from a sharpened version of the base model's distribution, a so-called power distribution, elicits comparable reasoning without additional training, curated datasets, or verifiers. However, making this method practical requires efficiently sampling from the power distribution. A sampler needs to "mix" to the power distribution, which necessitates moving between modes of the target distribution; intuitively, e.g., trying different reasoning strategies. The samplers proposed in prior works repeatedly select a "cut" position in the current reasoning trace uniformly at random and resample the suffix from that position onward. However, reasoning traces typically contain a few consequential decisions (e.g., the choice of proof strategy or algorithm), and we observe that a uniformly chosen cut tends to rewrite local details rather than revisit decision points. We introduce an algorithm (Entropy-Cut Metropolis-Hastings) that uses the base model's next-token entropy as a proxy to identify key decision points and resample from those positions. We empirically verify that entropy jumps are a useful proxy for decision points and, in a stylized model of reasoning, prove that our method's mixing time scales with the number of decisions in a trace rather than with the number of tokens, which can be much larger. Across MATH500, HumanEval, GPQA Diamond, and AIME26, our method consistently improves over baselines and RL-trained models.

Problem

• reasoning
• sampling
• power distribution
• decision points
• mixing time

Innovation

• Entropy-Cut
• Power Distribution Sampling
• Decision Point Identification
• Reasoning Efficiency
• Metropolis-Hastings