Generative Recursive Reasoning

2026-05-19

AI Summary

This work proposes a stochastic recursive reasoning framework that models inference as the evolution of probabilistic latent trajectories under a shared transition function, in contrast to existing deterministic approaches that follow a single hidden state path and thus struggle to support multi-hypothesis reasoning and flexible generation. By enabling parallel sampling of multiple trajectories within a recursive structure—achieved for the first time—the method integrates amortized variational inference to unify conditional reasoning and unconditional generation. Computational resources can be dynamically scaled by adjusting recursion depth and the number of sampled trajectories. Experiments demonstrate that the model significantly outperforms deterministic recurrent and recursive baselines on structured reasoning and multi-solution constraint satisfaction tasks, while also exhibiting strong generative capabilities.

Abstract

How should future neural reasoning systems implement extended computation? Recursive Reasoning Models (RRMs) offer a promising alternative to autoregressive sequence extension by performing iterative latent-state refinement with shared transition functions. Yet existing RRMs are largely deterministic, following a single latent trajectory and converging to a single prediction. We introduce \emph{Generative Recursive reAsoning Models (GRAM)}, a framework that turns recursive latent reasoning into probabilistic multi-trajectory computation. GRAM models reasoning as a stochastic latent trajectory, enabling multiple hypotheses, alternative solution strategies, and inference-time scaling through both recursive depth and parallel trajectory sampling. This yields a latent-variable generative model supporting conditional reasoning via $p_θ(y \mid x)$ and, with fixed or absent inputs, unconditional generation via $p_θ(x)$. Trained with amortized variational inference, GRAM improves over deterministic recurrent and recursive baselines on structured reasoning and multi-solution constraint satisfaction tasks, while demonstrating an unconditional generation capability. \href{https://ahn-ml.github.io/gram-website/}{https://ahn-ml.github.io/gram-website}

Problem

• Recursive Reasoning Models
• deterministic reasoning
• multi-trajectory computation
• probabilistic reasoning
• latent-variable generative model

Innovation

• Generative Recursive Reasoning
• Stochastic Latent Trajectory
• Multi-trajectory Computation
• Amortized Variational Inference
• Unconditional Generation