Probabilistic Tiny Recursive Model

Source

• Raw Markdown: paper_probabilistic-tiny-recursive-model-2026.md
• PDF: paper_probabilistic-tiny-recursive-model-2026.pdf
• Preprint: arXiv 2605.19943v1
• Official project page: Probabilistic Tiny Recursive Model
• Gonzo ML discussion: Telegram post 5663
• Gonzo ML Podcasts review: Telegram post 4279, continued through posts 4280–4282
• Gonzo-linked review: ArXivIQ summary

The exact root discussion is preserved at papers/probabilistic-tiny-recursive-model-2026/telegram-post-gonzo_ML-5663.md. The long review and its ordered continuation posts are preserved at papers/probabilistic-tiny-recursive-model-2026/telegram-post-gonzo_ML_podcasts-4279.md.

Status And Credibility

This is a current 2026 preprint, not a peer-reviewed archival result. arXiv lists only v1, submitted on 2026-05-19 under cs.AI, with a CC BY 4.0 license. The paper lists Amin Sghaier and Ali Parviz at Mila, with Sghaier also at ILLS and ETS Montreal, and Alexia Jolicoeur-Martineau as independent. Jolicoeur-Martineau is also the author of the base Tiny Recursive Model, which makes the method a credible direct extension rather than an unrelated reimplementation.

The official project page reproduces the method and headline tables, but says code is coming soon. No official PTRM implementation, checkpoint, peer-reviewed venue record, or author/lab X announcement was verified at ingest time. The experiments reuse standard pretrained TRM checkpoints, so the paper is reproducible in principle from the base model, but the new inference pipeline is not yet backed by a verified official release.

Core Claim

Probabilistic TRM (PTRM) turns deterministic Tiny Recursive Model inference into parallel stochastic search without retraining the model. It injects Gaussian noise into the latent reasoning state at every deep-recursion step, runs $K$ trajectories in parallel, and returns the candidate with the highest score from TRM’s existing Q head.

The important contribution is the separation of two inference-time compute axes:

• Depth: run more recursive refinement steps in one trajectory.
• Width: run more independently perturbed trajectories and select among them.

On the paper’s structured puzzle tasks, width usually adds more accuracy than extra depth and is more parallelizable. This is evidence that some deterministic TRM failures are proposal failures—the model can reach a correct basin under perturbation—rather than proof that the Q head is a universally reliable verifier or that tiny recursive models broadly outperform large language models.

Mechanism

For rollout $k$ and deep-recursion step $t$, PTRM perturbs the previous latent state and then applies the unchanged TRM recursion:

$${ϵ}_t^{(k)}\sim \mathcal{N}(0,{\sigma }^{2}I),z_{t-1}^{(k)}\leftarrow z_{t-1}^{(k)}+{ϵ}_t^{(k)},$$ $$\left(z_t^{(k)},y_t^{(k)}\right)=\mathrm{rec}\left(x,z_{t-1}^{(k)},y_{t-1}^{(k)}\right).$$

After $D$ steps, the output head decodes each answer and the Q head scores it:

$${\hat{y}}^{(k)}=\arg \max f_O\left(y_D^{(k)}\right),k^{\ast }=\arg \underset{k}{\max }{f}_Q\left(y_D^{(k)}\right).$$

The system returns ${\hat{y}}^{(k^{\ast })}$. The Q head is not a new zero-shot or label-free component: base TRM trained it with binary cross-entropy against exact answer correctness for adaptive computation. PTRM is training-free relative to that pretrained TRM checkpoint, but its selector inherits task supervision and calibration from TRM training.

The paper’s illustrative latent-state analysis is useful but narrow. On one previously failed puzzle, 8 of 100 noisy rollouts escape the observed bad basin and become correct while 92 remain trapped. This visualizes proposal coverage; it is not an estimate of a universal escape probability.

Evidence

Result	Exact protocol	Paper-grounded interpretation
PPBench golden aggregate rises from 62.6% to 91.2%.	Five reported puzzle types, 49 golden-set puzzles after omitting already-saturated Shakashaka; same 7M TRM-Att backbone; deterministic baseline $K=1,D=16$; PTRM best-Q@$K$ uses $K=100,D=48,\sigma =0.2$; metrics averaged over three seeds.	Strong controlled evidence for stochastic width plus Q selection on puzzle families represented in task-specific training. The test set is small and is not a broad reasoning benchmark.
Extra depth alone reaches 66.0% on the same PPBench slice.	Same checkpoint with $K=1,D=48$.	Most of the 28.6-point headline gain is not explained by simply tripling recursive depth.
On the larger PPBench validation set, best-Q@$K$ tracks pass@$K$ within one percentage point as $K$ grows.	Five puzzle types; three seeds; both metrics rise from 76.4% at $K=1$ to 89.5% at $K=100$ in the width-scaling study, while mode@$K$ gains only 1.3 points.	The pretrained Q head is a strong selector in this matched validation regime, and majority vote is a weaker selector.
Sudoku-Extreme reaches 98.75% best-Q@$K$ and 99.06% pass@$K$.	Standard reproduced TRM-MLP baseline 87.28%; $K=100,D=64,\sigma =0.3$; 5M parameters; three seeds.	Both proposal coverage and Q selection are strong on this exact checkable task.
Maze-Hard reaches 95.63% pass@$K$, but only 85.17% best-Q@$K$; mode@$K$ reaches 86.73%.	Standard reproduced TRM-Att baseline 83.80%; $K=100,D=16,\sigma =1.0$; 7M parameters; three seeds.	Noise finds substantial additional correct candidates, but the Q head cannot reliably identify them. Search and selection quality must be reported separately.
ARC-AGI-2 pass@1 rises from 7.36% to 8.47%.	$K=25,D=16,\sigma =0.2$ per data augmentation, then the standard augmentation-voting pipeline; pass@2 is unchanged at 9.72%, pass@100 rises from 14.31% to 15.97%.	The transfer is positive but small and still depends on task-specific augmentation and voting.
Q-guided Langevin gradients add no measurable gain over matched noise.	Appendix experiment replaces the linear Q head with an attention-pooled nonlinear head for tractable gradients, then compares Langevin dynamics with a noise-only ablation.	A useful negative result: the simple recurrent Gaussian perturbation, not gradient guidance, explains the observed gain in that experiment.

Cost And Baseline Calibration

The paper reports PTRM at approximately $0.001 per PPBench attempt and per correct solution, estimated from measured wall-clock time on one H100 priced at $2.50/hour. It compares this with $2.66 per attempt and $38.51 per correct solution for an ensemble of seven frontier LLMs. These ratios answer different questions:

• per attempt, the reported ratio is about 2,660×;
• per correct solution, it is about 38,510×;
• the abstract’s “less than $0.0001\times$ the cost” aligns with the per-correct framing, not with every possible denominator.

The LLM comparison is intentionally favorable to the ensemble in accuracy—it counts a puzzle solved if any model under any attempted strategy solves it and assumes a perfect verifier—but it is not an architecture-controlled comparison. PTRM is a task-specific 7M model trained from scratch on the relevant puzzle families; the LLMs are general-purpose systems with different training data, hidden reasoning harnesses, and selectively applied agentic retries. The result supports a specialized-system efficiency claim on PPBench, not a general claim that PTRM replaces frontier language models.

Gonzo Claim Audit

The root post and the longer podcast review correctly identify the core mechanism: recurrent Gaussian perturbations, parallel trajectories, and Q-head selection without PTRM-specific retraining. Several broader statements need narrower scopes.

Social framing	Paper-grounded reading
5–7M recursive models can outperform frontier LLMs on structured constraint-satisfaction tasks.	Supported on the reported PPBench golden subset under the paper’s protocol. It is not evidence for broad language reasoning, open-ended generation, or general deployment superiority.
Inference is more than 10,000× cheaper.	Supported only under the paper’s per-correct comparison: $0.001 versus $38.51. The per-attempt comparison is about 2,660×, and PTRM cost assumes one H100 at $2.50/hour.
The existing Q head is a zero-shot verifier.	It is reused without new PTRM training, but it was already trained on exact correctness labels in base TRM. It is well calibrated on PPBench/Sudoku and materially weaker on Maze-Hard and ARC-AGI-2.
PTRM needs practically no infrastructure.	Overstated. The paper uses one H100 80GB and up to 100 parallel rollouts. The reported dollar cost is low, but batching, memory, throughput, and deployment requirements remain real.
PTRM precedes and lays the basis for later GRAM-style models.	Not supported by chronology. PTRM and Generative Recursive Reasoning were both submitted to arXiv on 2026-05-19, and PTRM cites GRAM. They should be treated as concurrent neighboring approaches.
Recurrent perturbations are necessary for escaping local minima.	The paper shows that recurrent noise beats initial-state-only noise in the cited neighboring setup and unlocks correct candidates across its tasks. “Necessary” is too broad; other proposal mechanisms or trained stochastic dynamics may work.

Relation To Nearby Recursive And Fixed-Point Work

Source	Proposal mechanism	Selection / stopping signal	Training scope	Main boundary
PTRM	Isotropic Gaussian noise at every deep-recursion step of a pretrained deterministic TRM.	Pretrained correctness Q head selects one final candidate.	No PTRM retraining; selector was supervised during base TRM training.	Unique-answer puzzle tasks; selector calibration varies by task.
GRAM	Learned stochastic latent-variable transitions and parallel trajectory sampling.	Majority vote or learned Latent Process Reward Model.	Stochastic process is trained end to end through variational machinery.	Adds multi-solution coverage and unconditional generation, but remains puzzle-focused.
FRM	New discrete-flow proposals, self-conditioning, and verify-and-restart sampling.	Perturb-and-resolve stability of a completed candidate.	Flow model plus optional FlowDPO training.	Stability can rank checkable completions but is not general correctness or uncertainty.
FPRM	Additional recurrent depth with fixed-point optimization.	Hidden-state residual controls halting.	Looped model is trained for stable deep recurrence.	Convergence decides when to stop, not whether the answer is correct.

PTRM therefore adds a clean coverage-versus-selection decomposition to the local corpus. pass@K measures whether stochastic proposals reach a correct answer; best-Q@$K$ measures whether the internal selector can recover it. The Maze-Hard gap shows that more proposal diversity does not automatically yield more usable accuracy.

Foundation TSFM Relevance

Agenda slot	Verdict	Evidence	Missing pieces
Dynamic compute / inference-time scaling	adjacent	PTRM exposes parallel trajectory width as a budget knob alongside recursive depth.	Needs expected-FLOPs, wall-clock, memory, batching, and adaptive-$K$ tests on numeric time series or trajectories.
Latent-state refinement	adjacent	Controlled perturbations reveal that deterministic recursive state can be trapped in suboptimal basins.	Needs state diagnostics for noisy observations, irregular timestamps, exogenous variables, event streams, and nonstationary dynamics.
Multi-modal future distributions	warning	Multiple hidden trajectories are sampled, but the tasks have one correct answer and the method selects one candidate.	Needs calibrated coverage of several valid future trajectories without collapsing to the selector’s preferred mode.
Control and counterfactuals	insufficient evidence	Puzzles have constraints but no actions, control inputs, interventions, transition rewards, or downstream planning loop.	Needs action-conditioned next-state rollouts and decision-utility evaluation.
Benchmark level	warning	Strong gains are measured on checkable, mostly small-grid structured tasks with task-specific training.	Needs direct tests on numeric trajectories, long horizons, partial observability, rare regimes, and out-of-domain state transitions.

For multivariate time-series or world-model work, the transferable hypothesis is not “add noise and pick the highest confidence future.” It is to separate candidate-future coverage from candidate selection. A model could spend parallel compute on perturbed latent-state rollouts, then select using calibrated likelihood, constraints, ensemble agreement, downstream action value, or a learned utility head. Valid evaluation must report coverage before selection, calibration after selection, rare-regime recall, and realized serving cost. A Q-like head trained on one exact puzzle answer is not automatically a safe selector for stochastic or multi-modal physical and operational futures.

Limitations

• All positive evidence is on structured puzzles, and the largest headline PPBench result uses only 49 reported golden-set examples across five puzzle types.
• The method depends on pretrained TRM checkpoints and a Q head trained against exact correctness. It is not a general unsupervised verifier.
• Width scaling multiplies total computation by the number of rollouts even when parallel wall-clock latency is low. The paper does not establish a compute-optimal $K$ frontier across hardware, batch size, memory, and latency targets.
• Q-head calibration is task dependent. Maze-Hard has a 10.46-point gap between pass@$K$ and best-Q@$K$ at the reported setting.
• The LLM comparison mixes specialized task training with general-purpose models, provider-specific reasoning modes, selective agentic retries, and a hypothetical perfect ensemble verifier.
• The paper has no verified official code or PTRM-specific checkpoint release at ingest time.
• Sampling several latent trajectories and selecting one does not establish calibrated multi-modal generation, uncertainty estimation, causal state tracking, or action-conditioned world modeling.

Links Into The Wiki

• PTRM
• Tiny Recursive Model
• Generative Recursive Reasoning
• Flow Reasoning Models
• Fixed-Point Reasoners
• Looped Transformers And Test-Time Memory
• Efficient Recurrent Sequence Models
• Time-Series Scaling And Efficiency
• Foundation Time-Series Model Research Agenda
• Contradictions And Open Tensions

Open Questions

• Can rollout width be allocated adaptively from Q-head uncertainty, trajectory diversity, or marginal pass@$K$ gain instead of using a fixed $K$ for every input?
• What selector closes the large Maze-Hard proposal–selection gap without requiring an oracle checker?
• Does noise need to be isotropic and fixed across recursion depth, or should its covariance and scale depend on latent-state geometry and remaining compute?
• Under matched wall-clock and energy budgets, when does PTRM-style width beat deeper recurrence, GRAM-style learned stochasticity, FRM-style restart, ordinary ensembling, or energy-based candidate optimization?
• Can a time-series analogue preserve several valid regimes and rare transitions rather than selecting the most familiar or highest-density future?
• Which candidate-selection target is appropriate for action-conditioned world models: predictive likelihood, constraint satisfaction, calibrated risk, downstream control value, or a combination?