A Formal Comparison Between Chain-of-Thought and Latent Thought

2025-09-25

AI Summary

This work systematically compares Chain-of-Thought (CoT) and Latent Thought—two distinct reasoning paradigms for large language models. Addressing the lack of formal analysis on their mechanistic differences, we propose the first unified modeling framework that integrates recurrent Transformer architectures with continuous latent-space dynamics. Within this framework, we theoretically characterize and empirically evaluate both paradigms along three dimensions: computational parallelizability, decoding strategies, and task adaptability. Our analysis reveals that CoT relies on sequential stochastic sampling, making it suitable for complex reasoning tasks requiring approximate solutions; in contrast, Latent Thought enables fully parallelized, layer-wise evolution of implicit states, yielding superior computational efficiency and long-range dependency modeling. These findings provide a verifiable theoretical foundation for principled paradigm selection in model-based reasoning. The implementation is publicly available.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Chain-of-Thought (CoT) elicits reasoning in large language models by explicitly generating intermediate steps in natural language. In contrast, Latent Thought in looped models operates directly in the continuous latent space, enabling computation beyond discrete linguistic representations. While both approaches exploit iterative computation, their comparative capabilities remain underexplored. In this work, we present a formal analysis showing that Latent Thought in Looped Transformers enables parallel computation, which is more efficient than the inherently sequential process of CoT. In contrast, CoT leverages stochastic decoding to approximate solutions to problems where exact computation is intractable. These separations suggest the tasks for which depth-driven recursion is more suitable, thereby offering practical guidance for choosing between reasoning paradigms. Code is available at https://github.com/kevin671/cot-vs-loop.

Problem

• Compares reasoning efficiency between explicit and latent computation methods
• Analyzes parallel versus sequential processing in different reasoning paradigms
• Determines suitable tasks for depth-driven recursion versus stochastic decoding

Innovation

• Latent Thought enables parallel computation in looped models
• CoT uses stochastic decoding for intractable problem approximation
• Formal analysis compares depth-driven recursion versus sequential reasoning