Probabilistic Tiny Recursive Model

2026-05-19

AI Summary

Tiny recursive models often suffer from suboptimal local convergence due to insufficient exploration during inference. To address this limitation, this work proposes a task-agnostic test-time compute scaling method that injects Gaussian noise at each recursive step to generate multiple parallel reasoning trajectories. The model’s intrinsic Q-head is then leveraged to select the optimal output among these candidates, requiring neither retraining nor task-specific fine-tuning. This approach substantially enhances reasoning performance: on Sudoku-Extreme, accuracy improves from 87.4% to 98.75%, and it achieves 91.2% on the Pencil Puzzle Bench—significantly outperforming current large models (55.1%) despite using only 7 million parameters.

Abstract

Tiny Recursive Models (TRM) solve complex reasoning tasks with a fraction of the parameters of modern large language models (LLMs) by iteratively refining a latent state and final answer. While powerful, their deterministic recursion can lead to convergence at suboptimal solutions, without escape mechanism. A common workaround relies on task-specific input perturbations at test time combined with answer aggregation via voting. We introduce Probabilistic TRM (PTRM), a task-agnostic framework for test-time compute scaling that addresses this limitation through stochastic exploration. PTRM injects Gaussian noise at each deep recursion step, enabling parallel trajectories to explore diverse solution basins, and selects among them using the model's existing Q head (used for early stopping in the original TRM). Without requiring retraining or task-specific augmentations, PTRM enables substantial accuracy gains across benchmarks, including Sudoku-Extreme (87.4% to 98.75%) and on various puzzles from Pencil Puzzle Bench (62.6% to 91.2%). On the latter, PTRM achieves nearly double the accuracy of frontier LLMs (91.2% vs. 55.1%) at less than 0.0001x the cost, using only 7M parameters.

Problem

• Tiny Recursive Models
• deterministic recursion
• suboptimal convergence
• reasoning tasks
• escape mechanism

Innovation

• Probabilistic TRM
• stochastic exploration
• test-time compute scaling
• latent state refinement
• small-parameter reasoning