--- abstract: | How should future neural reasoning systems implement extended computation? Recursive Reasoning Models (RRMs) offer a promising alternative to autoregressive sequence extension by performing iterative latent-state refinement with shared transition functions. Yet existing RRMs are largely deterministic, following a single latent trajectory and converging to a single prediction. We introduce *Generative Recursive reAsoning Models (GRAM)*, a framework that turns recursive latent reasoning into probabilistic multi-trajectory computation. GRAM models reasoning as a stochastic latent trajectory, enabling multiple hypotheses, alternative solution strategies, and inference-time scaling through both recursive depth and parallel trajectory sampling. This yields a latent-variable generative model supporting conditional reasoning via $p_\theta(y \mid x)$ and, with fixed or absent inputs, unconditional generation via $p_\theta(x)$. Trained with amortized variational inference, GRAM improves over deterministic recurrent and recursive baselines on structured reasoning and multi-solution constraint satisfaction tasks, while demonstrating an unconditional generation capability. [https://ahn-ml.github.io/gram-website](https://ahn-ml.github.io/gram-website/) author: - | `\makebox[\textwidth][c]{% \textbf{Junyeob Baek}$^{1\dagger}$\thanks{Equal contribution} \quad \textbf{Mingyu Jo}$^{1\dagger*}$ \quad \textbf{Minsu Kim}$^{1,2}$ }`{=latex}\ \ `\makebox[\textwidth][c]{% \textbf{Mengye Ren}$^{3}$ \quad \textbf{Yoshua Bengio}$^{2,4}$ \quad \textbf{Sungjin Ahn}$^{1,3}$\thanks{Correspondence to: Junyeob Baek (\texttt{wnsdlqjtm@kaist.ac.kr}), Mingyu Jo (\texttt{mingyu.jo@kaist.ac.kr}), Sungjin Ahn (\texttt{sungjin.ahn@kaist.ac.kr})} }`{=latex}\ \ `\makebox[\textwidth][c]{$^1$KAIST \quad $^2$Mila -- Qu{\'e}bec AI Institute}`{=latex}\ `\makebox[\textwidth][c]{$^3$New York University \quad $^4$Universit{\'e} de Montr{\'e}al}`{=latex} bibliography: - ref_jy.bib title: Generative Recursive Reasoning --- ```{=latex} \DeclareMathOperator*{\argmax}{arg\,max} ``` ```{=latex} \newcommand\sj[1]{\textcolor{red}{{#1}}} ``` ```{=latex} \maketitle ``` # Introduction A central question for future neural reasoning systems is how extended computation should be implemented. Large autoregressive models typically scale reasoning by extending a sequence-generation process, whether intermediate computation is expressed explicitly as chain-of-thought tokens or implicitly in hidden or latent representations [@cot; @tot; @got; @coconut; @superposition; @continuous_cot]. A complementary direction is explored by Recursive Reasoning Models (RRMs), which use repeated computation to refine a persistent latent state rather than to append new elements to an output or reasoning sequence [@looped_transformer; @hrm; @trm]. This approach is appealing because it decouples reasoning depth from both parameter scale and output length: a compact model can perform many steps of internal computation by repeatedly applying shared transition functions over time. Recent recursive reasoning models such as HRM [@hrm] and TRM [@trm] provide early evidence for the potential of this approach in structured reasoning. Rather than producing a solution in a single feedforward pass, they perform extended computation through iterative latent-state refinement, deep supervision across refinement steps, and reasoning-oriented recurrent designs such as hierarchical latent dynamics. These features make them well suited to problems requiring constraint propagation, state tracking, iterative correction, and multi-step inference. More broadly, they build on a principle also explored in recurrent Transformer architectures such as Universal Transformers [@universal] and Looped Transformers [@looped_transformer]: shared Transformer blocks can be repeatedly applied to increase computational depth without increasing parameter count. Together, these models suggest that reasoning capability can emerge not only from scaling model size or generating longer traces, but also from the organization of computation itself. ![**Comparison of Latent Reasoning Trajectories.** Left: N-Queens Example with two valid solutions. Right: Given three independent runs for latent reasoning ($\tau_1, \tau_2, \tau_3$): (a) Prior RRMs (e.g. HRM, TRM) are deterministic---all runs collapse to an identical trajectory, converging to a single solution and failing to explore alternatives, while (b) GRAM explores diverse trajectories, producing diverse trajectories that reach multiple valid solutions $y_1$ and $y_2$, while naturally enabling parallel inference-time scaling.](figures/fig1_neurips.png){#fig:overview width="0.92\\linewidth"} While recurrent latent-state refinement provides an appealing mechanism for efficiently increasing reasoning depth, depth alone is not sufficient for many reasoning problems. A capable reasoning system should also be able to maintain uncertainty, consider alternative hypotheses, and explore multiple possible solution strategies [@kahneman2011thinking; @bengio2017consciousness]. This is especially important in settings where ambiguity or multiple valid solutions are intrinsic, and more generally in problems where a single refinement path may become trapped in a suboptimal reasoning trajectory. In this sense, future RRMs should be not only deep, in the sense of repeated refinement, but also wide, in the sense of maintaining and exploring multiple latent trajectories in parallel. Existing RRMs [@looped_transformer; @hrm; @trm; @universal], however, remain fundamentally deterministic: given the same input and initialization, they follow a single latent trajectory and converge to a single prediction. This deterministic recursion collapses the space of plausible reasoning paths into a single attractor, leaving probabilistic multi-hypothesis latent reasoning largely unexplored within the RRM paradigm. This motivates the central question of our work: *can recursive latent computation support probabilistic, generative, multi-hypothesis reasoning while preserving the efficiency of compact recurrent models?* In this paper, we propose Generative Recursive reAsoning Models (GRAM), a framework that turns recursive latent reasoning into probabilistic multi-trajectory computation. GRAM treats the reasoning process itself as a stochastic latent trajectory: at each recursion step, the model samples a transition conditioned on the input and the current reasoning state, rather than deterministically updating to a single next state. Repeating this process defines a distribution over possible reasoning trajectories, allowing the model to maintain multiple hypotheses, explore alternative solution strategies, and scale inference not only by increasing recursive depth but also by sampling trajectories in parallel. From a probabilistic perspective, GRAM is a latent-variable generative model: it models $p_\theta(y \mid x)$ by marginalizing over latent reasoning trajectories, while the same recursive process can also define an unconditional generative model $p_\theta(x)$ when the input is fixed or absent. We evaluate GRAM on controlled reasoning and generation tasks that serve as probes of the architectural properties targeted by our formulation: recursive refinement, stochastic exploration, multi-solution coverage, and inference-time scaling. Given this goal, our experiments focus on comparisons with the most relevant deterministic recurrent and recursive latent reasoning baselines, including Looped Transformers, HRM, and TRM, rather than frontier-scale general-purpose LLMs whose training data, inference budgets, and external scaffolding are not directly comparable. Sudoku-Extreme [@hrm] and ARC-AGI [@arc; @arc2] test structured reasoning under hard constraints and abstract transformations; N-Queens and Graph Coloring evaluate multi-solution recovery; and binarized MNIST [@mnist] probes the unconditional generative interpretation. Our main contribution is to establish probabilistic multi-trajectory recursion as a design principle for future recurrent and recursive reasoning architectures. Concretely, we make three contributions. First, we formulate recursive reasoning as a latent-variable generative process, where solutions are obtained by marginalizing over stochastic reasoning trajectories. Second, we introduce width-based inference-time scaling, enabling inference to scale not only with recursive depth but also with the number of sampled latent trajectories. Third, we provide empirical evidence that this formulation yields the intended architectural advantages over deterministic recurrent and recursive baselines, improving structured reasoning, multi-solution constraint satisfaction, and unconditional generation. # Generative Recursive Reasoning Models In this section, we introduce Generative Recursive reAsoning Models (GRAM), an instantiation of probabilistic recursive reasoning. We describe the architecture in `\cref{sec:architecture}`{=latex} and the training procedure in `\cref{sec:training}`{=latex}, with an architecture schematic shown in `\cref{fig:architecture}`{=latex}. ## Architecture {#sec:architecture} ```{=latex} \begin{wrapfigure}{r}{0.5\linewidth} \includegraphics[width=\linewidth]{figures/architecture9.pdf} \caption{\textbf{GRAM Architecture.} A single stochastic latent transition in the hierarchical instantiation $z=(h,l)$. After $K$ low-level refinements via $f_{\mathrm{L}}$, the high-level update $f_{\mathrm{H}}$ produces a deterministic proposal $u_t$, to which stochastic guidance $\epsilon_t$ is added: $h_t = u_t + \epsilon_t$.} \label{fig:architecture} \end{wrapfigure} ``` **Overview.** GRAM models the conditional distribution $p_\theta(y \mid x)$ by marginalizing over stochastic latent reasoning trajectories. Given an input $x$, GRAM first computes an embedding $$\begin{aligned} e_x = f_{\mathrm{enc}}(x;\theta), \end{aligned}$$ which is reused throughout the entire recursive computation. Starting from a fixed initial latent state $z_0$, the model evolves the latent state through learned stochastic transitions. The recursive computation is organized into two nested levels: inner and outer loops. At the inner level, a *latent transition* samples a new latent state conditioned on the previous latent state and the input embedding, $$\begin{aligned} z_{t} \sim p_{\theta}(z_t \mid z_{t-1}, e_x), \qquad t = 1, \dots, T. \label{eq:gram_transition} \end{aligned}$$ At the end of the $T$ transitions, the decoder produces a prediction, $\hat{y} = \argmax f_{\mathrm{dec}}(z_T;\theta)$. We refer to the sequence of $T$ transitions from the initial state $z_0$ to the final state $z_T$ as a *supervision step*. A supervision step is the unit at which the decoder is invoked, and the training objective is applied, with gradients computed as described in `\cref{sec:training}`{=latex}. At the outer level, $N_{\mathrm{sup}}$ supervision steps are applied recursively, with the final state of one supervision step serving as the initial state of the next, thereby forming the full recursive computation: $$\begin{aligned} z_0^{(1)} \;\xrightarrow{\;T\text{ transitions}\;}\; z_T^{(1)} = z_0^{(2)} \;\xrightarrow{\;T\text{ transitions}\;}\; \cdots \;\xrightarrow{\;T\text{ transitions}\;}\; z_T^{(N_{\mathrm{sup}})}, \label{eq:supervision_chain} \end{aligned}$$ where $z_t^{(n)}$ denotes the latent state at the $t$-th transition of the $n$-th supervision step, $z_0^{(1)}$ is the fixed initial state, and the terminal state of one supervision step serves as the initial state of the next ($\smash{z_0^{(n+1)} := z_T^{(n)}}$). This abstract formulation can be instantiated with various recurrent Transformer backbones, including flat designs such as Universal Transformers and Looped Transformers [@universal; @looped_transformer], as well as hierarchical designs such as HRM and TRM [@hrm; @trm]. **Stochastic Latent Transitions.** Unlike prior recursive reasoning models (RRMs) that update the latent state deterministically and follow a single fixed trajectory [@hrm; @trm], GRAM defines $p_\theta(z_t \mid z_{t-1}, e_x)$ as a stochastic transition, so that repeated computation induces a distribution over latent reasoning trajectories. Concretely, GRAM realizes this transition as a learned stochastic residual perturbation around a deterministic update: at each transition, the model first computes a deterministic update $u_t$ from $z_{t-1}$ and $e_x$, then samples a conditional perturbation from a state-dependent Gaussian, and adds it to $u_t$: $$\begin{aligned} \epsilon_t &\sim p_\theta(\epsilon_t \mid u_t) := \mathcal{N}\!\left(\mu_\theta(u_t),\, \sigma_\theta^2(u_t) I\right), \label{eq:gaussian_guidance} \\ z_t &= u_t + \epsilon_t. \end{aligned}$$ We refer to $\epsilon_t$ as the **learnable stochastic guidance**. The mean $\mu_\theta(u_t)$ encodes a state-dependent direction in which the trajectory is steered, while the variance $\sigma_\theta^2(u_t)$ controls the amount of exploration. This design allows GRAM to capture uncertainty, prevent convergence to local minima, and support robust exploration of the solution space without discarding the deterministic refinement performed by $u_t$. **Hierarchical Instantiation.** We instantiate the latent state with two interacting components, $z = (h, l)$. The high-level component $h$ is updated once per latent transition and carries abstract reasoning state, while the low-level component $l$ is updated $K$ times within a single transition and carries fine-grained intermediate computation. This decomposition separates the two roles across time scales, with $h$ accumulating slowly across transitions and $l$ refined rapidly within each one. With this hierarchical multi-scale structure, a single transition $z_{t-1} \to z_t$ is computed as follows. The low-level component is first refined for $K$ updates, with the high-level component held fixed: $$\begin{aligned} l_{t,k} = f_{\mathrm{L}}(h_{t-1},\, l_{t,k-1},\, e_x;\, \theta), \qquad k = 1, \dots, K, \label{eq:low_level} \end{aligned}$$ where $l_{t,0} := l_{t-1}$ and we write $l_t := l_{t,K}$ for the refined low-level component. The high-level component is then updated as a stochastic transition conditioned on the refined $l_t$, $$\begin{aligned} u_t &= f_{\mathrm{H}}(h_{t-1},\, l_t;\, \theta), \label{eq:deterministic_update} \\ \epsilon_t &\sim p_\theta(\epsilon_t \mid u_t) := \mathcal{N}\!\big(\mu_\theta(u_t),\, \sigma^2_\theta(u_t)\,I\big), \label{eq:gaussian_prior} \\ h_t &= u_t + \epsilon_t, \label{eq:high_level} \end{aligned}$$ and we set $z_t = (h_t, l_t)$. Note that stochasticity is introduced only at the high level: the low-level refinement is fully deterministic, while the stochastic guidance signal $\epsilon_t$ acts on the slower, more abstract component of the latent state, where it can steer the overall reasoning trajectory across transitions[^1]. Under this instantiation, the decoder reads only the high-level component, i.e., $f_{\mathrm{dec}}(z_T) = f_{\mathrm{dec}}(h_T)$. Additional architectural details are provided in Appendix `\ref{appx:arch_detail}`{=latex}. **Modeling Unconditional Distribution.** While the description so far focuses on the conditional setting $p_\theta(y \mid x)$, the same recursive process can also be defined as an unconditional generative model $p_\theta(x)$ when the input is replaced with an empty conditioning embedding. We use this formulation for generation tasks in `\cref{sec:generation}`{=latex}. ## Training {#sec:training} GRAM is trained to model the conditional distribution $p_\theta(y \mid x)$, where each training example consists of an input $x$ and its corresponding target $y$. As a probabilistic model, GRAM adopts a latent-variable formulation and is optimized by maximizing an evidence lower bound (ELBO) with respect to the generative parameters $\theta$ and variational parameters $\phi$. **Latent Variable Modeling.** We model GRAM as a latent-variable probabilistic model $p_{\theta}$, where the full latent trajectory $\tau = (z_0 \rightarrow \cdots \rightarrow z_{T_{\mathrm{Total}}})$ consists of a sequence of latent variables, with $T_{\mathrm{Total}} = T \times N_{\mathrm{sup}}$. The conditional likelihood is defined as $$p_{\theta}(y \mid x) = \int p_{\theta}(y \mid \tau, x)\, p_{\theta}(\tau \mid x)\, d\tau,$$ where $x$ denotes the input problem and $y$ denotes the corresponding ground-truth output. Direct maximum likelihood estimation of $\log p_{\theta}(y \mid x)$ is intractable due to the marginalization over latent trajectories. We therefore introduce a variational posterior $q_{\phi}(\tau \mid x,y)$ and optimize the evidence lower bound (ELBO), jointly training $\theta$ and $\phi$ via variational inference: $$\begin{aligned} \log p_{\theta}(y \mid x) \ge \mathbb{E}_{q_{\phi}(\tau \mid x,y)}[\log p_{\theta}(y \mid \tau, x)] - \mathrm{KL}\!\left( q_{\phi}(\tau \mid x,y) \,\|\, p_{\theta}(\tau \mid x) \right). \end{aligned}$$ During training, latent trajectories are sampled from the variational posterior $q_{\phi}(\cdot \mid x,y)$, which has access to both the input problem $x$ and the target output $y$. At inference time, where $y$ is unavailable, trajectories are instead generated from the learned prior $p_{\theta}(\cdot \mid x)$. Both the prior and the posterior are modeled as conditional Markov processes over latent states: $$p_{\theta}(\tau \mid x) = p(z_0)\prod_{t=1}^{T_{\mathrm{Total}}} p_{\theta}(z_t \mid z_{t-1}, x), \qquad q_{\phi}(\tau \mid x,y) = p(z_0)\prod_{t=1}^{T_{\mathrm{Total}}} q_{\phi}(z_t \mid z_{t-1}, x,y).$$ Here, $z_0$ is a fixed initial state shared by the prior and posterior. Both transitions are implemented by adding reparameterized Gaussian noise $\epsilon_t$ after a deterministic update $u_t$; the posterior uses the same transition module as the prior, but samples from a target-conditioned noise distribution $q_\phi(\epsilon_t \mid u_t, y)$, whereas the prior uses $p_{\theta}(\epsilon_t \mid u_t)$. Since the two processes share the same Markov structure and all stochasticity is introduced through $\epsilon_{1:T_{\mathrm{Total}}}$, their trajectory distributions can be equivalently represented in noise space. Moreover, since GRAM decodes the output only from the terminal latent state, the likelihood term satisfies $p_{\theta}(y \mid \tau,x)=p_{\theta}(y \mid z_{T_{\mathrm{Total}}},x)$. Therefore, the full trajectory-level ELBO can be written as $$\begin{aligned} \mathcal{L}_{\mathrm{ELBO}} = \mathbb{E}_{q_{\phi}} \big[ \log p_{\theta}(y \mid z_{T_{\mathrm{Total}}}, x) \big] - \sum_{t=1}^{T_{\mathrm{Total}}} \mathbb{E}_{q_{\phi}(\epsilon_{