On the (Intuitionistic) Logic of Next-Token Prediction

2026-08-08

AI Summary

This work formalizes the next-token prediction mechanism of large language models from a logical perspective. By leveraging intuitionistic implication logic and the Curry–Howard isomorphism, it models autoregressive prediction as hypothetical inference and views sequence generation as the stepwise extension of a constructive proof. The study establishes, for the first time, a theoretical connection between next-token prediction and intuitionistic logic, revealing the logical relationship between sequential order and prediction granularity. Building on this insight, it introduces a novel neural architecture grounded in proof theory. The proposed model, integrating a Prolog theorem prover with a multiplicative RNN, is evaluated on key properties including commutative and non-commutative reasoning as well as single- and multi-token prediction. Furthermore, the framework elucidates intrinsic connections among Transformers, state space models, and recurrent LLMs within a unified logical setting.

Abstract

We model in intuitionistic implicational logic the key enabler of today's GenerativeAI: the next-token prediction in autoregressive causal neural networks. In our framework, next-token prediction corresponds to modus ponens, and sequence processing becomes constructive proof extension under the Curry-Howard correspondence. Our Prolog-based specialized theorem provers validate fundamental properties of the neural models, among which relations between commutative vs. non-commutative sequencing and single-token vs. multi-token prediction choices. We derive a neural architecture equivalent to multiplicative RNNs that arises naturally from a proof-theoretic interpretation of next-token prediction as nested intuitionistic implication and position the model relative to transformers, state-space models and recursive LLMs.

Problem

• next-token prediction
• intuitionistic logic
• autoregressive models
• Curry-Howard correspondence
• generative AI

Innovation

• intuitionistic logic
• next-token prediction
• Curry-Howard correspondence
• multiplicative RNNs
• constructive proof