ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spaces

2026-02-12

AI Summary

This work addresses the instability of existing implicit reasoning methods across diverse scenarios, where low-confidence reasoning paths often introduce noise and lead to high-confidence errors. To mitigate this, the authors propose a confidence-aware dynamic routing mechanism that adaptively selects between continuous latent-space reasoning for high-confidence contexts and discrete symbolic reasoning for low-confidence ones, enabling the first synergistic switching between these two paradigms. By integrating large language model–based confidence estimation, latent-space inference, and discrete chain-of-thought generation, the approach effectively suppresses error propagation and corrects flawed reasoning. Experimental results demonstrate consistent improvements across multiple STEM and programming benchmarks, with an average 19.70-point gain in Pass@1 accuracy, up to a 15.55% reduction in output length, and significantly enhanced calibration of reasoning confidence.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Recent work explores latent reasoning to improve reasoning efficiency by replacing explicit reasoning trajectories with continuous representations in a latent space, yet its effectiveness varies across settings. Analysis of model confidence dynamics under latent reasoning reveals that thinking trajectories ending in incorrect answers contain fewer low-confidence steps than those ending in correct answers. Meanwhile, we suggest that soft embeddings aggregated by multiple low-confidence thinking alternatives may introduce and propagate noise, leading to high confidence in unreliable reasoning trajectories. Motivated by these observations, ThinkRouter, an inference-time confidence-aware routing mechanism is proposed to avoid high confidence and noise for efficient reasoning. ThinkRouter routes thinking to the discrete token space when model confidence is low, and to the latent space otherwise. Extensive experiments on STEM reasoning and coding benchmarks across diverse large reasoning models demonstrate that ThinkRouter outperforms explicit CoT, random routing, and latent reasoning baselines in terms of accuracy, achieving an average improvement of 19.70 points in Pass@1, while reducing generation length by up to 15.55%. Further comprehensive analysis reveals that ThinkRouter can calibrate errors arising from explicit CoT and latent reasoning, and accelerates end-of-thinking token generation by globally lowering model confidence.

Problem

• latent reasoning
• reasoning efficiency
• model confidence
• noise propagation
• discrete reasoning

Innovation

• ThinkRouter
• latent reasoning
• confidence-aware routing
• discrete-continuous hybrid reasoning
• efficient inference