No Global Plan in Chain-of-Thought: Uncover the Latent Planning Horizon of LLMs

2026-02-02

AI Summary

This study investigates whether large language models possess global planning capabilities during chain-of-thought (CoT) reasoning. By introducing Tele-Lens—a probing method that dynamically analyzes internal model representations across multiple tasks—the work systematically reveals, for the first time, that the reasoning process primarily relies on local, incremental transitions rather than global planning. Building on this insight, the authors propose a novel hypothesis: uncertainty over the entire CoT path can be effectively captured by representations at just a few local positions. Leveraging this principle, they develop an approach to automatically identify dispensable CoT steps without incurring performance degradation. Experiments demonstrate that uncertainty estimates derived from only a small subset of CoT positions accurately reflect overall reasoning uncertainty, providing strong evidence for the models’ myopic planning behavior.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Reasoning under Uncertainty: Uncertainty Representations
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

This work stems from prior complementary observations on the dynamics of Chain-of-Thought (CoT): Large Language Models (LLMs) is shown latent planning of subsequent reasoning prior to CoT emergence, thereby diminishing the significance of explicit CoT; whereas CoT remains critical for tasks requiring multi-step reasoning. To deepen the understanding between LLM's internal states and its verbalized reasoning trajectories, we investigate the latent planning strength of LLMs, through our probing method, Tele-Lens, applying to hidden states across diverse task domains. Our empirical results indicate that LLMs exhibit a myopic horizon, primarily conducting incremental transitions without precise global planning. Leveraging this characteristic, we propose a hypothesis on enhancing uncertainty estimation of CoT, which we validate that a small subset of CoT positions can effectively represent the uncertainty of the entire path. We further underscore the significance of exploiting CoT dynamics, and demonstrate that automatic recognition of CoT bypass can be achieved without performance degradation. Our code, data and models are released at https://github.com/lxucs/tele-lens.

Problem

• Chain-of-Thought
• Large Language Models
• latent planning
• planning horizon
• reasoning

Innovation

• Chain-of-Thought
• latent planning
• Tele-Lens
• uncertainty estimation
• reasoning horizon