THOUGHTTERMINATOR: Benchmarking, Calibrating, and Mitigating Overthinking in Reasoning Models

2025-04-17

AI Summary

Large reasoning models frequently suffer from “overthinking,” generating excessive redundant tokens—particularly on simple tasks—thereby compromising both inference efficiency and reliability due to poor calibration. Method: We introduce a difficulty-aware token allocation benchmark, construct the lightweight easy-task dataset DUMB500, and propose THOUGHTTERMINATOR—a training-free, black-box decoding method. For the first time, we quantitatively uncover a power-law relationship between task difficulty and optimal token consumption, enabling dynamic, cross-difficulty calibration. Contribution/Results: Evaluated across mathematics, logic, and code generation, THOUGHTTERMINATOR reduces redundant token usage by over 40% on easy tasks for mainstream reasoning models, while simultaneously improving accuracy by 1.2–2.8 percentage points. The approach significantly enhances calibration fidelity and inference efficiency without architectural modification or fine-tuning.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Natural Language Processing: (Large) Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Reasoning models have demonstrated impressive performance on difficult tasks that traditional language models struggle at. However, many are plagued with the problem of overthinking--generating large amounts of unnecessary tokens which don't improve accuracy on a question. We introduce approximate measures of problem-level difficulty and demonstrate that a clear relationship between problem difficulty and optimal token spend exists, and evaluate how well calibrated a variety of reasoning models are in terms of efficiently allocating the optimal token count. We find that in general, reasoning models are poorly calibrated, particularly on easy problems. To evaluate calibration on easy questions we introduce DUMB500, a dataset of extremely easy math, reasoning, code, and task problems, and jointly evaluate reasoning model on these simple examples and extremely difficult examples from existing frontier benchmarks on the same task domain. Finally, we introduce THOUGHTTERMINATOR, a training-free black box decoding technique that significantly improves reasoning model calibration.

Problem

• Measuring relationship between problem difficulty and optimal token usage
• Evaluating calibration of reasoning models on easy and hard problems
• Introducing THOUGHTTERMINATOR to improve reasoning model calibration

Innovation

• Measures problem difficulty and optimal token spend
• Introduces DUMB500 dataset for easy question calibration
• Proposes THOUGHTTERMINATOR decoding technique