SuperThoughts: Reasoning Tokens in Superposition

2026-06-11

AI Summary

This work addresses the high computational cost of chain-of-thought (CoT) reasoning, which stems from its token-by-token generation process, as well as the training instability and limited scalability of existing continuous latent-space approaches. The authors propose SuperThoughts, a framework that compresses every two consecutive CoT tokens into a single latent representation and incorporates a lightweight multi-token prediction (MTP) module to decode two tokens per step. By integrating discrete supervision signals with a confidence-aware fallback mechanism, SuperThoughts reduces reasoning length by 20–30% on benchmarks such as MATH500 and AMC while nearly doubling inference throughput, with only a marginal 1–2 percentage point drop in accuracy.

Abstract

Long Chain-of-Thought (CoT) reasoning improves LLM problem-solving but is computationally expensive due to sequential token generation. While recent works explore reasoning in continuous latent spaces to bypass discrete token generation, they often struggle with training stability and fail to scale to complex, long-horizon tasks due to lack of supervision signal. We propose SuperThoughts, which compresses pairs of consecutive CoT tokens into single latent representations and decodes two tokens per step via a lightweight Multi-Token Prediction (MTP) module. This preserves discrete token supervision at training time while doubling throughput at inference time. We finetune Qwen2.5-Math-1.5B-Instruct, Qwen2.5-Math-7B-Instruct, Qwen2.5-Math-14B-Instruct, and evaluate on MATH500, AMC, OlympiadBench, and GPQA-Diamond. With a confidence-based adaptive mechanism that falls back to standard decoding when uncertain, SuperThoughts achieves $\sim$20--30\% CoT length reduction while maintaining accuracy with minimal degradation (1-2 points accuracy drop on most tasks).

Problem

• Chain-of-Thought reasoning
• computational efficiency
• token generation
• training stability
• long-horizon tasks

Innovation

• SuperThoughts
• Chain-of-Thought
• Multi-Token Prediction
• Latent Representation
• Inference Acceleration