Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate

2026-06-15

AI Summary

Existing language models struggle to dynamically determine when, what type of, and how much computational resource to allocate for implicit reasoning during inference. This work proposes a typed and budget-aware implicit reasoning framework that, at each step of autoregressive decoding, dynamically chooses between generating text or invoking specialized implicit computation modules, thereby supporting global planning, local state updates, and reusable procedural abstractions. To the best of our knowledge, this is the first approach to enable dynamic scheduling and typed management of implicit reasoning, jointly optimizing the timing, type, and resource allocation of computations. Experiments demonstrate that the framework improves accuracy by up to 14.49 points over Chain-of-Thought (CoT) and by 4.30 points over the strongest baseline across three prominent large language models, while exhibiting strong generalization and the lowest forgetting rate.

Abstract

Chain-of-thought (CoT) prompting improves reasoning in large language models (LLMs) by externalizing intermediate computation as discrete text tokens, but this textual interface also introduces redundancy and inference overhead. Latent reasoning offers a promising alternative by carrying part of the computation in continuous representations. However, existing methods typically predefine when latent computation is invoked and how it is allocated during decoding, leaving a key problem unresolved: when to invoke latent computation, what type of computation to perform, and how much budget to allocate. We propose \textbf{Ty}ped \textbf{L}at\textbf{e}nt \textbf{R}easoning (Tyler), a typed and budget-aware framework for latent reasoning during autoregressive decoding. Tyler learns a policy that, at each decoding step, chooses between emitting a text token and switching to a latent computation module specialized for a particular reasoning function. Once invoked, an operator maps the current reasoning state into latent tokens that support global planning, local state updates, or reusable procedural abstraction. Across extensive experiments on three backbone LLMs, Tyler improves accuracy by up to 14.49 points over CoT and by up to 4.30 points over the strongest competing baseline. It further generalizes across diverse reasoning domains and achieves the best final-stage performance with the lowest forgetting.

Problem

• latent reasoning
• chain-of-thought
• computation allocation
• reasoning policy
• autoregressive decoding

Innovation

• Latent Reasoning
• Typed Computation
• Budget-aware Policy
• Autoregressive Decoding
• Chain-of-Thought