Beyond the Frontier: Stochastic Backtracking for Efficient Test-Time Scaling

2026-05-24

AI Summary

This work addresses the trade-off between improving reasoning accuracy and reducing token consumption in test-time scaling for language models. The authors propose a stochastic backtracking mechanism based on a persistent history prefix pool, which enables the reasoning process to revisit prior states and thereby avoids premature commitment and diversity collapse commonly observed in frontier expansion. The approach integrates sub-pool selection with Power Backtrack Sequential Monte Carlo (SMC), leveraging PRM scores and hybrid correction weights to guide exploration. Experimental results across multiple mathematical reasoning benchmarks and varying model scales demonstrate that the method achieves comparable or superior accuracy with significantly fewer generated tokens, outperforming existing PRM-guided approaches.

Abstract

Test-time scaling improves language model reasoning by spending additional compute to explore multiple solution trajectories. The key challenge is to maximize accuracy while minimizing the total number of generated tokens during reasoning. Recent PRM-guided methods score intermediate prefixes to steer this search, but most are frontier-only: they keep only the current active prefixes and irreversibly prune or resample away the rest using noisy PRM scores. This can cause premature commitment, diversity collapse, and the loss of prefixes that still admit correct continuations. We introduce stochastic backtracking over a persistent pool of historical prefixes, allowing test-time compute to revisit previously generated states instead of only expanding the current frontier. To make this efficient, we propose two complementary mechanisms. Subpool Selection strengthens greedy PRM-guided search by applying Top-N selection within random subpools, giving historical prefixes a chance to bypass over-scored frontier candidates. Power Backtrack Sequential Monte Carlo extends SMC-style resampling to the persistent pool using powered PRM scores and mixture-corrected weights. Across mathematical reasoning benchmarks and model scales, our methods consistently achieve higher accuracy per token count, and the same level of accuracy using only a fraction of the token count in comparison to strong PRM-guided baselines, demonstrating that persistent-pool stochastic backtracking provides a simple and effective way to improve the accuracy-token trade-off in test-time scaling.

Problem

• test-time scaling
• stochastic backtracking
• prefix pruning
• accuracy-token trade-off
• reasoning efficiency

Innovation

• stochastic backtracking
• test-time scaling
• persistent prefix pool
• PRM-guided search
• Sequential Monte Carlo