Funnel of Thoughts: Efficient Test-Time Scaling via Early Voting and Rollout Pruning

2026-08-15

AI Summary

This study addresses the high computational costs and inefficient trajectories associated with multi-sampling in large reasoning models by proposing FoT, a training-free trajectory pruning method leveraging lexical signals. By dynamically identifying and eliminating invalid paths through cues such as hesitation tokens, FoT enables cross-architecture transferability without additional training. Experimental results demonstrate that FoT maintains the accuracy of 32-trajectory voting while reducing attention FLOPs by half, generation FLOPs by 56.1%, and inference latency by 37.6%. These findings indicate significant improvements in test-time computational efficiency, establishing FoT as a novel paradigm for the efficient and reliable deployment of large reasoning models.

Abstract

Large Reasoning Models produce diverse, sometimes inconsistent answers across repeated queries on the same problem, so multi-sample inference is a prerequisite for reliable deployment. Majority voting at k rollouts is the standard solution and the de facto accuracy target for this regime, but it is prohibitively expensive at the scale LRMs require. We introduce Funnel of Thoughts (FoT), an inference-time method that preserves the full 32-trajectory voted accuracy while halving its attention FLOPs, a 28.8% reduction in full-model inference cost. Across 115K reasoning trajectories from six LRMs, we find that unproductive trajectories often reveal themselves through repeated hesitation markers such as "Wait", "Actually", and "perhaps." These trajectories are less likely to reach the correct answer and consume disproportionate attention FLOPs, degenerating into no-answer loops in the worst case. Built on this training-free lexical signal, FoT identifies the vocabulary that captures these pathological patterns and prunes affected trajectories before completion, reducing online generation attention FLOPs by 56.1% and wall time by 37.6% without any additional model inference; the same signal transfers without retuning across held-out architectures and out-of-domain tasks.

Problem

• Test-Time Scaling
• Large Reasoning Models
• Multi-sample Inference
• Inference Cost
• Rollout Pruning

Innovation

• Test-Time Scaling
• Rollout Pruning
• Early Voting
• Hesitation Markers
• Training-Free