--- title: "Flow Reasoning Models: Scaling Reasoning Through Iterative Self-Refinement" author: - Alec Helbling - Andrey Bryutkin - Mauro Martino - Nima Dehmamy - Hendrik Strobelt affiliations: - Georgia Tech - MIT - MIT-IBM Computing Research Lab - IBM Research abstract: | Discrete flow models have recently shown promising performance on few-step text generation; however, when naively applied to structured reasoning tasks such as Sudoku and Zebra puzzles, they converge confidently to incorrect answers (solving only $\sim36\%$ of Sudoku puzzles). We introduce Flow Reasoning Models (FRMs), a training and test-time-scaling framework for structured reasoning with flow models. We make the observation that, despite their poor solve rate, flow models can act as their own verifiers. A correct answer is a stable fixed point of the denoising dynamics, returning to itself when re-noised and re-solved. This enables a test-time-scaling paradigm: propose many candidate solutions and keep those that are dynamically stable, which alone reaches high solve rates on Sudoku ($100\%$) and Zebra ($95.9\%$). This even generalizes to harder out-of-distribution puzzles like Sudoku-Extreme ($96.1\%$), without ever training on that distribution. This pure search, however, wastes a great deal of computation generating incorrect candidate solutions. We therefore design a training recipe to improve the base model's efficiency. First, we train flow models with a self-conditioning channel and close it at inference, letting them refine their own past predictions. Second, we train models to avoid their own failed generations using direct preference optimization. These changes substantially improve the base model's efficiency, letting it reach $99.2\%$ on Sudoku in just $7$ forward passes, over $8\times$ fewer than the strongest matched masked-diffusion baseline we compare needs for the same accuracy. When combined with test-time scaling, this lets flow models solve hard out-of-distribution puzzles (e.g. Sudoku-Extreme) far more efficiently. --- # Abstract Discrete flow models have recently shown promising performance on few-step text generation; however, when naively applied to structured reasoning tasks such as Sudoku and Zebra puzzles, they converge confidently to incorrect answers (solving only $\sim36\%$ of Sudoku puzzles). We introduce Flow Reasoning Models (FRMs), a training and test-time-scaling framework for structured reasoning with flow models. We make the observation that, despite their poor solve rate, flow models can act as their own verifiers. A correct answer is a stable fixed point of the denoising dynamics, returning to itself when re-noised and re-solved. This enables a test-time-scaling paradigm: propose many candidate solutions and keep those that are dynamically stable, which alone reaches high solve rates on Sudoku ($100\%$) and Zebra ($95.9\%$). This even generalizes to harder out-of-distribution puzzles like Sudoku-Extreme ($96.1\%$), without ever training on that distribution. This pure search, however, wastes a great deal of computation generating incorrect candidate solutions. We therefore design a training recipe to improve the base model's efficiency. First, we train flow models with a self-conditioning channel and close it at inference, letting them refine their own past predictions. Second, we train models to avoid their own failed generations using direct preference optimization. These changes substantially improve the base model's efficiency, letting it reach $99.2\%$ on Sudoku in just $7$ forward passes, over $8\times$ fewer than the strongest matched masked-diffusion baseline we compare needs for the same accuracy. When combined with test-time scaling, this lets flow models solve hard out-of-distribution puzzles (e.g. Sudoku-Extreme) far more efficiently. # Introduction {#sec:intro} Diffusion and flow-based models dominate continuous modalities such as images and audio, but have lagged behind autoregressive models on language. Recent masked diffusion models [@lou2024discretediffusionmodeling; @sahoo2024simpleeffectivemaskeddiffusion] have shown promising performance on structured reasoning tasks, where a globally consistent answer must be produced rather than a locally plausible next token. However, the capabilities of flow models for structured reasoning remain under-investigated. We applied discrete flow models [@potaptchik2026discreteflowmaps; @lee2026flowmaplanguagemodels] to constraint satisfaction problems such as Sudoku and logic puzzles, and found their performance to be poor: sampled naively, a flow model trained on Sudoku solves only about $36\%$ of puzzles. Yet despite these failures, we find that the denoising dynamics of discrete flows expose an internal signal that distinguishes correct from incorrect solutions. Concretely, a correct solution is a stable fixed point of the denoising process: re-noising it and resolving returns the same answer, whereas many incorrect states are unstable and change. Reading this stability lets the model identify its own correct solutions far more reliably than its sampler produces them, a *generation--verification gap*. This gap gives a direct strategy for test-time scaling: rather than trust a single sample, we propose many candidate solutions and keep the one the model's own stability signal accepts. Scaling this search lifts our constraint-satisfaction tasks to high solve rates --- [Sudoku]{.smallcaps} ($100\%$), [Zebra]{.smallcaps} ($95.9\%$) --- and even generalizes to out-of-distribution [Sudoku-Extreme]{.smallcaps} ($96.1\%$), which the model was never trained on. This search is effective but expensive, since reaching high accuracy can require many candidates per problem. ![**Flow reasoning models find correct solutions at stable fixed points.** **(a)** The dynamics of discrete flow models identify the correctness of samples: a correct solution sits in a stable basin and returns consistently to itself after perturbations, while error states occupy unstable basins that drift away when re-resolved. **(b)** We perform reasoning through iterative refinement with self-conditioning, where feeding the model's previous logits $\ell_{\text{prev}}$ back as conditioning drives each attempt toward a stable fixed point. **(c)** Combining this with self-verification using the model's internal signal of correctness is a powerful test-time-scaling approach that saturates Sudoku-Extreme examples, achieving 97% despite being trained only on much easier Sudoku-Shah puzzles. Grids are drawn as $4\times4$ schematics for clarity; the actual Sudoku and Sudoku-Extreme tasks are full $9\times9$ puzzles.](figures/crown_jewel.png){#fig:crown-jewel width="\\linewidth"} We improve the base model in two complementary ways, at inference and in training. First, we modify the flow model to revise its own past predictions at inference time through self-conditioning [@chen2023analogbitsgeneratingdiscrete], feeding each step the model's previous answer so a single attempt iteratively refines toward a stable fixed point. This gives the flow model a form of memory across refinement steps, connecting it to recurrent-depth and looped transformers [@geiping2025latentrecurrentdepth; @solvetheloop2026; @movahedi2026fixedpointreasoners] that likewise iterate a computation toward a fixed point. FRMs differ in what they read from that iteration: completed flow-LM states are re-noised and re-solved, making stability a label-free verifier at test time, while the training objective uses train-time gold/self-mined pairs to reshape the same fixed-point geometry. Second, we train the model to correct its own mistakes with a direct preference objective we call `\flowdpo{}`{=latex}, which pushes the model away from its self-generated incorrect solutions and toward correct ones, more targeted than the maximum-likelihood objective flow models are usually trained with. Together, these make the model a far more efficient proposer: it reaches $99.2\%$ on Sudoku in just $7$ forward passes, over $8\times$ fewer than the strongest masked-diffusion baseline, with consistent gains on Zebra as well. ```{=latex} \setlength{\leftmargini}{1.2em} ``` - **Flow Reasoning Models for structured reasoning.** We turn discrete flow language models into stateful solvers for checkable tasks by running them as self-conditioned fixed-point iterations rather than one-shot parallel samplers (Section `\ref{sec:method-verify}`{=latex}). - **Self-conditioned refinement with verifier-free restarts.** Feeding each step the model's previous prediction lets a single attempt refine over time; a stability readout then accepts stable solutions or re-noises and refines again, yielding test-time scaling without an external verifier (Algorithm `\ref{alg:self-refine}`{=latex}). - **Fixed-point stability as an internal correctness signal.** Re-noising a candidate, resolving it, and measuring drift gives a high-AUROC signal: correct solutions are robust under the model's dynamics, while many incorrect ones drift away. We use this signal to reject spurious states on hard out-of-distribution data such as [Sudoku-Extreme]{.smallcaps} (Section `\ref{sec:method-probe}`{=latex}). - **`\flowdpo{}`{=latex} reshapes the same fixed-point landscape.** This pairwise objective contrasts gold solutions with self-mined confident mistakes at the wrong tokens, outperforming cross-entropy-only controls and more than doubling single-shot Sudoku pass@1 ($35.8\!\rightarrow\!80.6\%$ in the audited EMA-reference regime; Section `\ref{sec:exp-training}`{=latex}). # Flow Reasoning Models {#sec:method-verify} A *flow reasoning model* is a conditional discrete flow language model run not as a one-shot sampler but as an iteration that carries *memory* of its own running prediction. At each denoising step the model is conditioned on the answer it currently believes in (self-conditioning, §`\ref{sec:method-selfcond-loop}`{=latex}), so generation becomes a dynamical system whose state is the model's evolving guess. The premise is simple: on the checkable tasks we study, correct completed states behave as stable fixed points of that system, while many mistakes are brittle under small perturbations. Reasoning, on this view, is iteration toward a stable fixed point, a lens shared with recurrent-depth and attractor accounts of reasoning [@geiping2025latentrecurrentdepth; @solvetheloop2026]. ```{=latex} \begin{figure*}[t] \includegraphics[width=\linewidth]{figures/sudoku_rollout.pdf} \caption{\textbf{Self-conditioning refines one Sudoku attempt to the correct grid.} From the same initial noise, each panel feeds back the previous prediction as self-conditioning; wrong cells (red) fall from 28~$\to$~26~$\to$~3~$\to$~0. No restart or external verifier is used.} \label{fig:sudoku-rollout} \end{figure*} ``` This section sets up the object, then acts on its fixed points in two ways. First, we *drive* the dynamics with self-conditioning, so a single attempt can refine itself. Second, we *read* the dynamics by re-noising a completed state and checking whether it returns. The next section uses the same fixed-point view to *reshape* the landscape with `\flowdpo{}`{=latex}. ## The flow reasoning model framework {#sec:method-selfcond} #### A discrete flow language model. A flow language model [@lee2026flowmaplanguagemodels] generates discrete sequences by running a continuous flow in an embedding of the vocabulary: a sequence of $L$ tokens $w=(w_1,\dots,w_L)$, $w_i\in\mathcal{V}$, is lifted to a continuous tensor $x_1=(e(w_1),\dots,e(w_L))\in\mathbb{R}^{L\times d}$ by a token embedding $e:\mathcal{V}\to\mathbb{R}^{d}$ and read back by a nearest-embedding (argmax) decode. Gaussian noise $\varepsilon\sim\mathcal{N}(0,I)$ at $t{=}0$ is joined to data at $t{=}1$ along the linear interpolant $$x_t=(1-t)\,\varepsilon+t\,x_1, \label{eq:path}$$ (the stochastic-interpolant / straight-path construction; the noise schedule and citations are in Appendix `\ref{app:flowmap}`{=latex}); sampling integrates the learned flow from a noise draw at $t{=}0$ to a clean embedding at $t{=}1$ in a handful of Euler steps and decodes it to tokens. Closing the self-conditioning loop over these steps (below) is what makes this sampler the dynamical system whose fixed points the rest of the section analyzes. #### Training: a tokenwise classifier under cross-entropy. The network is the *denoiser*, the conditional expectation of the clean endpoint given the noisy state, $$D_t(x) := \mathbb{E}\big[x_1 \mid x_t = x\big],\qquad v_t(x) = \frac{D_t(x) - x}{1-t}, \label{eq:denoiser}$$ where the velocity $v_t$ of the linear interpolant `\eqref{eq:path}`{=latex} is recovered from $D_t$ in closed form. Because $x_1$ is a token embedding, this expectation is the posterior over the vocabulary at each position. We implement it by predicting logits $\ell_\theta(x_t,t)\in\mathbb{R}^{L\times|\mathcal{V}|}$ and a token distribution $q_\theta(\cdot\mid x_t,t)=\mathrm{softmax}(\ell_\theta(x_t,t))$ on the simplex [@lee2026flowmaplanguagemodels]. The embedding expectation induced by $q_\theta$ gives $D_\theta$, hence the velocity in Eq. `\eqref{eq:denoiser}`{=latex}. We train the token distribution with cross-entropy on the true tokens at every noise level, $$\mathcal{L}_{\mathrm{CE}}(\theta)=\mathbb{E}_{t,\,w,\,\varepsilon}\Bigl[ -\sum_{i=1}^{L}\log q_\theta(w_i\mid x_t,t)\Bigr], \label{eq:ce}$$ so a single network call yields the token distribution, the velocity (via Eq. `\eqref{eq:denoiser}`{=latex}), and the readout $\hat{w}_i=\arg\max_{w}q_\theta(w\mid x_t,t)$. Matching this categorical posterior with cross-entropy, rather than regressing an embedding velocity with an $L^2$ loss, is what makes flow language models work [@potaptchik2026discreteflowmaps]. This objective teaches the dynamics and creates the per-token log-scores that the verifier and `\flowdpo{}`{=latex} will later reuse. ```{=latex} \begin{figure*}[t] \includegraphics[width=\linewidth]{figures/tts_results.png} \caption{\textbf{Self-verification enables test-time scaling across tasks.} (Left) Our \textsc{FRM} training scheme saturates \textsc{Sudoku} (Shah) in a single pass, and test-time scaling allows even a poor base model to saturate the task after several rounds. (Center) On hard out-of-distribution \textsc{Sudoku-Extreme} data, our self-verification test-time-scaling algorithm solves ${\sim}100\%$ of the tasks; our \textsc{FRM} training regime further improves the efficiency of training, and at the high end the base model overtakes the \textsc{FRM}-trained model. (Right) This approach also generalizes to the \textsc{Zebra} puzzle task, where we see scaling improve the ceiling of performance on both models, and \textsc{FRM} improves both the efficiency and the ceiling.} \label{fig:tts-results} \end{figure*} ``` #### A conditional denoiser. Reasoning is *conditional*: a subset of positions is given as clue tokens $c$, and the model must fill the remaining answer positions $a$. We write the completed assignment as $y=(c,a)$ and let $\mathcal{M}$ denote the positions over which losses or stability scores are averaged. For Sudoku, these are the answer cells not fixed by the clue; clue cells are clamped to their given values, so including them would not change exact-match evaluation. We turn the flow language model into a conditional, masked generator by holding clue positions clean at every noise level and clamping them throughout sampling, so the denoiser predicts the answer cells from the conditional distribution $$q_\theta\big(a_i \mid x_t,\, c,\, t\big), \label{eq:cond-denoiser}$$ and the cross-entropy of Eq. `\eqref{eq:ce}`{=latex} is taken only over $\mathcal{M}$. A *solution* is correct if the task checker accepts the completed assignment. The method uses that checker for training and evaluation only; at test time it substitutes the model's own fixed-point stability. ## Learning to Self-Refine at Inference Time {#sec:method-selfcond-loop} We first drive the dynamics, making a single solve build on its own previous prediction. A plain flow sampler produces a fresh denoising estimate at each Euler step, but the next step does not explicitly know what the model just believed. Self-conditioning supplies that short-term memory [@chen2023analogbitsgeneratingdiscrete]: the logit network receives a second input $s$ carrying the previous prediction, $$\ell_\theta\big(x_t,\,t,\,s\big),\qquad s = \ell_{\text{prev}}, \label{eq:selfcond}$$ where $s{=}0$ recovers the plain model. The channel is zero-initialized, so an untrained model is unchanged; with self-conditioning, $q_\theta(\cdot\mid x_t,t,s)= \mathrm{softmax}(\ell_\theta(x_t,t,s))$. Training and representation details are in Appendix `\ref{app:selfcond-details}`{=latex}. Closing this feedback across Euler steps turns sampling into an *inner refinement loop*. Starting from noise $x_{t_0}{=}\varepsilon$ with clue cells clamped and $\ell^{(0)}{=}0$, each call emits logits $$\begin{aligned} \ell^{(j)} &= \ell_\theta\big(x_{t_j},\,t_j,\,\ell^{(j-1)}\big), & \hat{y}^{(j)} &= \arg\max_{w} q_\theta\big(w\mid x_{t_j},t_j,\ell^{(j-1)}\big), \label{eq:selfcond-pred}\\ x_{t_{j+1}} &= x_{t_j} + (t_{j+1}-t_j)\,v_\theta\big(x_{t_j},t_j,\ell^{(j-1)}\big), \label{eq:selfcond-euler} \end{aligned}$$ and the logits become the conditioning input for the next step. We commit when the decoded assignment stops changing, $\hat{y}^{(j)}{=}\hat{y}^{(j-1)}$. This contraction fixes many brittle mistakes within a single attempt; stable wrong attractors can still remain, which motivates the readout and outer loop below. In distribution, this drive-only mechanism is already strong: *supervised training alone*, with no preferences, mined negatives, or test-time verifier, lifts [Sudoku]{.smallcaps} from a $\sim$$38\%$ one-shot solve rate to $97.8\%$ in a handful of steps (§`\ref{sec:exp-selfcond}`{=latex}). ## Self-Verification Through Fixed-Point Stability {#sec:method-probe} ![**Re-noising separates correct from incorrect cells.** The candidate grid is re-noised to interior time $t$ and re-solved; we read per-cell re-solution discrepancy $\Delta\mathrm{CE}$. (a) Correct cells stay near zero under large re-noising, while incorrect cells deviate steeply. (b) On a single grid, the discrepancy concentrates on the model's wrong cells (red) and stays near zero elsewhere.](figures/perturbation_stability.png){#fig:basin width="\\linewidth"} The same fixed-point view gives a verifier. On the tasks we study, correctness correlates with stability, so we can ask whether a candidate returns to itself after perturbation. This readout matters most when the inner loop settles on a spurious fixed point on hard out-of-distribution puzzles: iterating longer does not escape it, but a stability probe can reject it and trigger a fresh attempt. #### Reading stability from the denoising dynamics. A generated solution $y$ is a near-fixed-point of the model. To test its stability, we re-noise its endpoint $x_1(y)$ to an interior time $t$, $\tilde{x}_{t,r}(y)=(1-t)\varepsilon_r+t\,x_1(y)$, then re-solve with a fresh pass of the sampler. Writing $F_\theta(\cdot,t,1;c)$ for the sampler/integrator from time $t$ to $1$, $$T_{t,r}(y;c) := F_\theta(\tilde{x}_{t,r}(y),t,1;c),$$ where $F_\theta(\cdot,t,1;c)$ integrates from time $t$ to $1$ with clue cells clamped to $c$ and the self-conditioning channel reset. In the reported experiments we fix $t{=}0.40$ and average over $k$ re-noise draws: $$S_{t,k}(y) = \frac{1}{k}\sum_{r=1}^{k} d_{\mathcal{M}}\big(y,\,T_{t,r}(y;c)\big), \label{eq:stability}$$ using the *renoise cross-entropy* distance $$d_{\mathcal{M}}\big(y,\,T_{t,r}(y;c)\big) = -\frac{1}{\lVert \mathcal{M}\rVert_1}\sum_i \mathcal{M}_i\, \log q_\theta^{t\to1}\big(y_i \mid \tilde{x}_{t,r}(y),c\big), \label{eq:renoise-ce}$$ where $q_\theta^{t\to1}$ denotes the token distribution read from the resolved logits of the re-solve. This is the same simplex cross-entropy used in training (Eq. `\eqref{eq:ce}`{=latex}), now used to score return to the candidate rather than fit a label. The estimate saturates at a single draw, so the score is cheap to query. Lower $S_{t,k}$ means a more stable fixed point. Figure `\ref{fig:basin}`{=latex} shows the cell-level geometry. Correct cells remain stable under re-noising; confident mistakes drift, causing their re-solution cross-entropy to rise. Thus the readout tells us not only whether a grid is unstable, but which cells carry the instability, the signal the wrong-cells objective in Section `\ref{sec:method-flowdpo}`{=latex} later trains on. #### Why the signal is not circular. $S_{t,k}$ re-reads the model's own logits, so one might ask whether it merely repackages the generator's confidence. It does not, because generation and verification ask different questions of the same network. Generating must put enough mass on one exact grid among many competitors; verifying asks whether a given grid is a stable local attractor. A state can therefore be easy to recognize as stable ($S_{t,k}$ low) even when the sampler rarely lands on it. This decoupling is measured directly: on [Sudoku-Extreme]{.smallcaps}, renoise-CE selection stays near $1.0$ ($0.98$--$1.00$, Table `\ref{tab:selection}`{=latex}) while single-shot solve rate falls to $\sim1\%$ (Section `\ref{sec:exp-verify}`{=latex}). This is not oracle pass@$N$: the score never consults the gold answer at test time. #### Closing the outer loop: verify on convergence, re-noise on reject. Inner and outer loop are two operations on the same fixed-point structure: the inner loop *contracts* a trajectory onto a fixed point, and the outer loop detects when that fixed point is spurious. We run self-conditioning to convergence, score the attempt with $S_{t,k}$, and accept if $S_{t,k}<\tau$; otherwise we re-noise from scratch, clear self-conditioning, and refine again under a fixed forward-pass budget. Algorithms `\ref{alg:selfcond-sample}`{=latex} and `\ref{alg:self-refine}`{=latex} state this procedure, charging every denoising and verifier pass so the compute axis in Section `\ref{sec:experiments}`{=latex} counts the true cost. #### From reading to reshaping. {#sec:method-selection} Reading stability across independent attempts also gives a passive best-of-$N$ selector: keep the most stable proposal, with no external verifier or ground truth (Appendix `\ref{app:propose-verify}`{=latex}). But reading alone cannot change where the model places probability mass. Cross-entropy can leave a specific wrong completion in its own confident basin, while renoise-CE identifies the cells where that completion is unstable. The next section uses the same fixed-point/locality view for training: `\flowdpo{}`{=latex} deepens the correct basin against the model's self-mined wrong fixed points. ```{=latex} \begin{figure*}[t] \captionof{lstlisting}{\textbf{Self-conditioning sampling (the inner loop).} Drive one attempt onto a fixed point by feeding the model its own prediction at each step.} \label{alg:selfcond-sample} \begin{lstlisting}[language=Python,aboveskip=2pt,belowskip=0pt] # flow_model.sample(cond) eps = randn(shape) z = clamp(eps, cond) sc = None num_passes = 0 for t in euler_schedule(): logits = flow_model(z, t, self_cond=sc) pred = argmax(logits) sc = logits z = euler_step(z, logits, t) num_passes += 1 return pred, num_passes \end{lstlisting} \hfill \captionof{lstlisting}{\textbf{Iterative self-refinement (the outer loop).} Accept a stable fixed point, else re-noise and refine again under a forward-pass budget $B$.} \label{alg:self-refine} \begin{lstlisting}[language=Python,aboveskip=2pt,belowskip=0pt,escapeinside={(*@}{@*)}] # self_refine(flow_model, cond, tau, B, k) spent = 0 while spent < B: pred, num_passes = flow_model.sample(cond) spent += num_passes score = flow_model.score(pred, cond, k) spent += verifier_nfe(k) if score < tau: return pred # budget exhausted: return the last attempt return pred (*@@*) \end{lstlisting} \end{figure*} ``` # `\flowdpo{}`{=latex}: Preference Training on Self-Mined Mistakes {#sec:method-flowdpo} Reading stability can reject a wrong fixed point after a solve, but it does not make the next solve less likely to enter that basin. `\flowdpo{}`{=latex} turns the same fixed-point view into training: given a gold completed assignment $y^+$ and a confident wrong completion $y^-$ mined from the current model, it deepens the basin around $y^+$ relative to that specific competitor. Cross-entropy gives the model useful basins, but it is a marginal objective: it raises the gold token at each cell and suppresses alternatives only indirectly through the softmax normalizer. It does not name the particular wrong completion the model already occupies. A model can therefore become more confident on gold while leaving a nearby wrong fixed point intact, separated from gold by only the few cells whose values invalidate the grid. `\flowdpo{}`{=latex} instead applies the direct preference loss [@rafailov2024directpreferenceoptimizationlanguage] to the pair $(y^+,y^-)$, using the flow model's train-matched denoising score in the $\log\pi$ slot and no auxiliary verifier. #### The contrast acts only on the decisive cells. The fixed-point view also fixes where the contrast applies. Two competing completed states usually coincide except on the cells that decide correctness, so we restrict the preference to the gold-supervised wrong-cell mask $W_i=\mathcal{M}_i\mathds{1}[\,y_i^-\neq y_i^+\,]$. A whole-sequence contrast would push down many tokens the negative already gets right, fighting the cross-entropy term that should keep them; localizing to $W$ removes that conflict, and the two configurations differ sharply in practice (Section `\ref{sec:exp-training}`{=latex}). The renoise-CE score is the test-time analogue of this locality signal; during training, gold supplies the exact mask. `\flowdpo{}`{=latex} scores the gold and mined negative under the same corrupted negative state, so the contrast asks a local question: on the cells where this wrong fixed point disagrees with gold, should the flow denoiser prefer $y^+$ or $y^-$? The resulting descent step raises the gold tokens on $W$ and pushes down the mined wrong tokens on those same cells, the targeted update ordinary cross-entropy does not explicitly perform. The exact masked score, per-negative margin, and expectation over noise draws are in Appendix `\ref{app:flowdpo-details}`{=latex}. Figure `\ref{fig:flowdpo-histogram}`{=latex} sketches the intended basin-level effect on one held-out puzzle. ```{=latex} \begin{figure*}[t] \includegraphics[width=\linewidth]{figures/flowdpo_histogram.pdf} \caption{\textbf{\flowdpo{} deepens the correct basin against the model's own confident mistakes.} For one held-out Sudoku puzzle, the base model (left) places the gold grid (highlighted) as only a weak, dilute mode over many competing states, whereas \flowdpo{} (right) collapses most mass onto it (gold share $26\%\!\to\!68\%$, distinct grids $139\!\to\!75$). This is an illustrative single-puzzle sketch; aggregate pass@1 gains are in Table~\ref{tab:flowdpo-ablation}.} \label{fig:flowdpo-histogram} \end{figure*} ``` ## Mining the model's own mistakes {#sec:method-mine} The contrast needs competing mistakes to downweight, and the model supplies them. For each clue $c$ we draw many conditional completions from the model itself, with clue tokens clamped to gold, check each completion against gold, and keep the distinct confident wrong answers as $\mathcal{Y}^-(c)=\{y^-_1,\dots,y^-_n\}$. Each negative is paired with the same gold completion $y^+$, and gold identifies the exact wrong-cell mask $W$. Each instance yields a variable-size negative set, so the loss averages the pairwise contrast over the distinct wrong completions mined for that clue. The source matters: random off-distribution grids often sit in no competing basin and produce little training signal, whereas self-mined hard negatives raise Sudoku pass@1 the most (Section `\ref{sec:exp-training}`{=latex}), echoing on-policy over off-policy preference data [@tajwar2024preference]. Mining sampler, deduplication, and replay buffer details are in Appendix `\ref{app:flowdpo-details}`{=latex}. ## Training loop {#sec:method-loop} The pieces close into a self-improvement loop: the flow model mines its own confident mistakes and updates against them with the wrong-cells preference contrast, then mines again from the improved model. Reference choice is important for stability: the reference must keep the training target anchored. Re-snapshotting it to the current policy can let the target chase the policy, whereas an exponential-moving-average reference pins the endpoint used in our reported ablation. The experiments test the three structural predictions: self-mined negatives beat random ones, wrong-cell support beats whole-sequence contrast, and an EMA-pinned reference keeps the target anchored (Section `\ref{sec:exp-training}`{=latex}). The remaining levers ($K$ negatives per group, the priority replay buffer, update cadence, and pseudocode) are in Appendix `\ref{app:flowdpo-details}`{=latex}. # Experiments {#sec:experiments} The experiments follow the abstract's read-first arc. We first *read* the dynamics, asking whether renoise-CE recognizes correct fixed points even when the sampler rarely reaches them (Section `\ref{sec:exp-verify}`{=latex}). We then *drive* those dynamics with self-conditioning and verify-and-restart, testing whether the readout becomes an efficient solver (Section `\ref{sec:exp-selfcond}`{=latex}). Finally, we *reshape* the landscape with `\flowdpo{}`{=latex}, asking whether training makes the proposer less likely to enter self-mined wrong basins (Section `\ref{sec:exp-training}`{=latex}). ## Experimental setup {#sec:setup} We study two structured-reasoning task families plus an out-of-distribution Sudoku split. For each task family we train a conditional flow language model on (clue, answer) pairs; for every reported number we state the sampling budget (number of Euler steps or verify-and-restart rounds) used to produce it. Pass@1 means one sampled completion scored by the task checker: exact match for both Sudoku and Zebra. Whenever a result uses a verifier, restart, or a different integration-step budget, we state that budget explicitly and charge every forward pass, including the verifier's re-solve. Architecture and hyperparameters are in Appendix `\ref{app:setup}`{=latex}. #### Datasets. [Sudoku]{.smallcaps}: complete a $9\times9$ grid ($L{=}81$ over digits $1$--$9$) from a revealed clue subset; we train a $\sim30$M-parameter model on the @shah2024causallanguagemodelingelicit set and evaluate both in-distribution and on the harder, difficulty-rated [Sudoku-Extreme]{.smallcaps} split, an evaluation-only out-of-distribution benchmark the model is never trained on. [Zebra]{.smallcaps}: recover a full attribute-by-house assignment grid from relational clues, scored by exact match. We use the Zebra puzzles released by @shah2024causallanguagemodelingelicit (the same source as our [Sudoku]{.smallcaps} data, not the ZebraLogic benchmark). The `\flowdpo{}`{=latex} additive ablations and test-time-scaling curves use a $4\times4$ layout; the self-conditioning [FRM]{.smallcaps} single-shot numbers (Figure `\ref{fig:gap}`{=latex}) use a pinned $5\times5$ layout. Full specifications and the all-size Zebra diagnostic are in Appendix `\ref{app:setup}`{=latex}. ## Flow models can verify their own correctness {#sec:exp-verify} ```{=latex} \begin{figure*}[t] \includegraphics[width=\linewidth]{figures/verification_gap.pdf} \caption{\textbf{A generation--verification gap, and closing it with training.} (Left) Models are far better at distinguishing correct from incorrect states through renoise-CE than they are at naively generating correct solutions, as indicated by the high AUROC ($\approx$$1.0$) against single-shot solve rates of only $\sim$$11$--$41\%$. (Right) Training the models with \flowdpo{} and self-conditioning helps close this gap, significantly raising one-shot solve rates: \textsc{Sudoku} $40.5{\to}98.0\%$, \textsc{Sudoku-Extreme} $10.7{\to}27.4\%$, and \textsc{Zebra} ($5\times5$) $39.8{\to}71.3\%$.} \label{fig:gap} \end{figure*} ``` #### Reading stability recognizes correct states far better than the sampler reaches them. Renoise-CE perturbs a finished state and asks whether it returns, so its accuracy directly reads fixed-point stability. On pools where the rare correct state is injected, that readout is near-perfect: AUROC is $\approx1.0$ on [Sudoku]{.smallcaps} and [Sudoku-Extreme]{.smallcaps} and $0.999$ on [Zebra]{.smallcaps}, while the same base samplers solve only $\sim11$--$41\%$ of single shots (Figure `\ref{fig:gap}`{=latex}). This is the generation--verification gap: the dynamics encode which states are stable far better than the sampler lands on them. Read as a *selector*, the same signal ranks the correct state first almost whenever the pool contains one (Table `\ref{tab:selection}`{=latex}; top-$1$ accuracy $\approx$$1.0$ even out of hundreds of [Sudoku-Extreme]{.smallcaps} candidates), while whole-state plurality vote [@wang2023selfconsistency] tops out near $0.69$--$0.84$. The next section uses this selector to turn recognition into test-time scaling. ## Self-conditioning improves one-shot efficiency {#sec:exp-selfcond} A recognition gap is only worth having if we can act on it. The most direct way is to *drive* the dynamics onto a stable fixed point with the self-conditioning inner loop, which on the in-distribution split is nearly sufficient by itself: *supervised training only*, with no preferences, no mined negatives, and no test-time verifier, lifts single-shot Sudoku from $\sim$$38\%$ to $97.8\%$ (6-seed mean, in a handful of steps), as the closed loop contracts each attempt onto the wide-basin fixed point the gap says the model can recognize. ## Self-verification enables test-time scaling {#sec:exp-tts} ```{=latex} \tablestyle{4pt}{1.2} ``` ::: {#tab:tts-scaling} Sudoku-Extreme ------------------------------------------------- ------------------------------------------ ---------- ---------- ---------- ---------- ---------- ---------- 2-8 `\footnotesize `{=latex}Number of rounds Method $1$ $16$ $64$ $256$ $1024$ $4096$ $8192$ Base $+$ Self-Verification Scaling 10.7 34.8 50.8 76.0 **92.3** **97.4** **98.6** [FRM]{.smallcaps} $+$ Self-Verification Scaling **27.4** **48.1** **62.2** **76.8** 88.9 94.1 96.1 : **Self-verification scales at test time, saturating out-of-distribution [Sudoku-Extreme]{.smallcaps} with enough compute.** Columns give the number of verify-and-restart *rounds*; each round proposes one candidate (a $128$-step inner loop) and verifies it with the renoise-CE self-verifier, restarting until accepted. Both the base flow model and [FRM]{.smallcaps} (self-conditioning $+$ `\flowdpo{}`{=latex}) climb toward the coverage limit ($96$--$99\%$) on these out-of-distribution puzzles, despite training only on much easier [Sudoku]{.smallcaps} (Shah). [FRM]{.smallcaps} is markedly more efficient at low budget---reaching a given solve rate in far fewer rounds---while at high budget the two become comparable, as the base flow model's extra rounds eventually close the gap. The in-distribution [Sudoku]{.smallcaps} (Shah) counterpart, where self-conditioning alone is essentially sufficient, is in Table `\ref{tab:tts-scaling-shah}`{=latex}. ::: Out of distribution the inner loop can instead settle on a confident but wrong (spurious) fixed point, and here the *read* closes the loop: the renoise probe detects the instability, and the outer loop re-noises and refines again. Selection errors are not the dominant bottleneck in these pools (Section `\ref{sec:exp-verify}`{=latex}): reading stability ranks the correct state first almost whenever the pool contains one, even out of hundreds of [Sudoku-Extreme]{.smallcaps} candidates. Verify-and-restart therefore spends compute escaping spurious basins, lifting [Sudoku-Extreme]{.smallcaps} substantially under large forward-pass budgets. Scaling the proposal pool instead of restarting (best-of-$N$ selection, Eq. `\eqref{eq:selection}`{=latex}) reaches the same ceiling as the purely passive read; both charge every forward pass, including the verifier's re-solve, and both are detailed in Appendix `\ref{app:tts}`{=latex}. #### Reading stability beats masked diffusion at far fewer forward passes. {#sec:exp-mdm} The same stability readout is most compute-efficient on [Sudoku]{.smallcaps} (Shah et al.) at a matched $30$M-parameter budget when run in the NFE-matched regime: a $4$-step self-conditioned proposer with verifier-gated restart and a charged renoise-CE verifier draw. Against the masked-diffusion (MDM) family (MDM/MDLM [@sahoo2024simpleeffectivemaskeddiffusion], its remasking sampler ReMDM [@wang2026remaskingdiscretediffusionmodels], and an adaptive confidence-ordered decode [@kim2025trainworstplanbest], each at its characteristic number of forward passes), the strongest baseline reaches $99.2\%$ at $\sim$$57$ NFE. Reading fixed-point stability, verify-and-restart reaches $99.2\%$ at a mean of $\sim$$7$ NFE, matching it with $\sim$$8\times$ fewer forward passes (Table `\ref{tab:mdm-comparison}`{=latex}); even single-shot, self-conditioning already reaches $88.0\%$ at $4$ NFE and $97.1\%$ at $8$. The compared MDM baselines are strong samplers, but they do not expose this train-matched stability readout; the self-conditioning flow model does. ```{=latex} \tablestyle{6pt}{1.15} ``` ::: {#tab:mdm-comparison} Method Forward Passes (NFE) Solve Rate (%) $\uparrow$ --------------------------------------------------- ---------------------- --------------------------- MDM [@sahoo2024simpleeffectivemaskeddiffusion] $256$ $24.2$ ReMDM [@wang2026remaskingdiscretediffusionmodels] $256$ $31.7$ Adaptive MDM [@kim2025trainworstplanbest] $57$ $99.2$ `\ours `{=latex}**Flow Reasoning Model (Ours)** $\mathbf{7}$ $\mathbf{99.2}$ : **A Flow Reasoning Model solves Sudoku as accurately as the strongest masked-diffusion methods, at a fraction of the compute.** Matched $30$M-parameter models on [Sudoku]{.smallcaps} (Shah et al.); accuracy comes from the $4$-step self-conditioning proposer with verifier-gated restart and a charged renoise-CE check, which reaches $99.2\%$ in $\sim$$7$ forward passes where the best masked-diffusion baselines need $57$. ::: ## Contrasting self-mined mistakes improves over maximum likelihood {#sec:exp-training} ```{=latex} \begin{figure*}[t] \begin{subfigure}[t]{0.175\linewidth} \includegraphics[width=\linewidth]{figures/flow_dpo_ablation_labels.pdf} \end{subfigure} \hfill \begin{subfigure}[t]{0.261\linewidth} \includegraphics[width=\linewidth]{figures/flow_dpo_ablation_sudoku.pdf} \end{subfigure} \hfill \begin{subfigure}[t]{0.261\linewidth} \includegraphics[width=\linewidth]{figures/flow_dpo_ablation_sudoku_extreme.pdf} \end{subfigure} \hfill \begin{subfigure}[t]{0.261\linewidth} \includegraphics[width=\linewidth]{figures/flow_dpo_ablation_zebra_4x4.pdf} \end{subfigure} \caption{\textbf{A relative objective deepens the correct basin.} Single-shot solve rate (pass@1, mean $\pm$ SEM on held-out splits; $4$ seeds) as the cumulative \flowdpo{} ablation adds random preference pairs, wrong-cell support, hard negatives from the live model, and an EMA-pinned reference. Tasks are Sudoku \citep{shah2024causallanguagemodelingelicit}, the evaluation-only out-of-distribution \textsc{Sudoku-Extreme} split, and Zebra ($4\times4$). This is the 1024-step reshaping ablation, distinct from the low-step self-conditioning regime of Section~\ref{sec:exp-selfcond}.} \label{fig:flowdpo-ablation} \end{figure*} ``` The fixed-point argument motivates three ablations. Random negatives should be weak because they often name no occupied competing basin; whole-sequence contrast should waste gradient away from the decisive cells; and a drifting reference should chase the policy instead of anchoring the target geometry. We test these predictions by measuring single-shot pass@1 after each structural constraint is introduced (Figure `\ref{fig:flowdpo-ablation}`{=latex}, $4$ seeds, held-out split; per-arm numbers in Table `\ref{tab:flowdpo-ablation}`{=latex}). A preference objective on *random* negatives barely moves Sudoku ($35.8\!\to\!35.9$). Restricting the contrast to exactly the negative's wrong cells, where the two fixed points part, is the *largest single increment* ($35.9\!\to\!69.3$ on Sudoku, $+33.4$). Mining hard negatives on-policy adds a further increment by surfacing the spurious fixed points the model actually occupies, and an EMA-pinned reference adds the last audited Sudoku endpoint ($80.6$) while making the training target less likely to be chased. Durability is task-dependent, so we pin the reference and expose both grid- and token-level contrasts rather than claim one universally robust setting. #### It is the explicit contrast, not the supervision, that deepens the basin. Holding the wrong-cells support fixed and varying only the learning signal (Table `\ref{tab:flowdpo-arms}`{=latex}), plain supervised fine-tuning (*Regular SFT*) and even SFT restricted to exactly the negative's wrong cells (*SFT on Wrong Cells*, which raises the gold token there through ordinary cross-entropy) move the baseline by at most $4.4$ points, whereas adding an explicit reference-ratio term that names the mined wrong state as the loser raises Sudoku by ${\sim}31$ ($36.4\!\to\!67.0$). This is the structural prediction made concrete: *SFT on Wrong Cells* can improve the gold logits on the disputed cells, but it never names the mined wrong completion as the state to suppress. The gain is not hard-example supervision but the contrastive suppression of the model's own confident mistakes, exactly what marginal maximum likelihood does not target. #### A deeper basin reaches each coverage level with a smaller pool. Deepening the correct basin should also make the few-step sampler land in it more often, and it does: `\flowdpo{}`{=latex} reaches the same $\sim$$100\%$ ceiling with a smaller proposal pool at every level (Appendix `\ref{app:selection-dpo}`{=latex}; e.g. Shah $N^{*}{=}16{\to}8$), reconcentrating mass onto reachable solutions rather than adding new ones, while its near-perfect selection is undisturbed by the concentration (top-$1$ accuracy $\sim$$1.0$ even at its own smaller pools). The fixed-point structure is sharpened, not blurred. #### The recipe transfers beyond in-distribution Sudoku. The same wrong-cells `\flowdpo{}`{=latex} update lifts single-shot pass@1 on the harder tasks (Figure `\ref{fig:flowdpo-ablation}`{=latex}): on the out-of-distribution [Sudoku-Extreme]{.smallcaps} split (the same model, never trained on it) it climbs $10.4\!\to\!20.7$, and on [Zebra]{.smallcaps} ($4\times4$) $52.1\!\to\!88.5$ across the cumulative ablation, consistent with the in-distribution Sudoku gain. On Zebra the weak base has enough headroom that even random preference pairs already help, but the wrong-cells contrast is what carries the climb to its endpoint. (This ablation isolates the `\flowdpo{}`{=latex} update alone: the DPO ladder evaluated at $1024$ Euler steps, *without* self-conditioning, so its endpoints differ from the full single-shot [FRM]{.smallcaps} numbers of Figure `\ref{fig:gap}`{=latex}, which add the self-conditioning inference loop.) # Related Work {#sec:related} #### Iterative fixed points and diffusion attractors. Reasoning-by-iteration appears in recurrent-depth language models that spend test-time compute by reapplying a latent block [@geiping2025latentrecurrentdepth], attractor and equilibrium models that solve for a converged state [@solvetheloop2026; @bai2019deepequilibrium], and looped Transformer reasoners that use fixed-point convergence for adaptive halting on tasks such as Sudoku and Maze [@movahedi2026fixedpointreasoners]. Diffusion analyses likewise describe generation as motion through attractor landscapes, including collapse onto training examples [@biroli2024dynamicalregimes], associative-memory basins [@pham2025memorizationgeneralization], and ultrastable cyclic-denoising attractors [@sharma2026cyclicdenoising]. FRMs apply fixed-point structure to self-conditioned flow language models on checkable completions, using perturbed re-solves for generation, label-free correctness readout, and training against spurious basins. #### Flow and masked-diffusion language models. Flow language models [@lee2026flowmaplanguagemodels] connect flow matching [@lipman2023flowmatchinggenerativemodeling; @albergo2023stochasticinterpolants] with simplex-aware discrete flows [@potaptchik2026discreteflowmaps]. Continuous-embedding neighbors such as ELF [@hu2026elfembeddedlanguageflows] and LangFlow [@chen2026langflow] show that continuous denoising can be competitive for text, but they target open-ended language modeling rather than checkable reasoning. Masked and discrete diffusion LMs [@sahoo2024simpleeffectivemaskeddiffusion; @nie2025llada] are the nearest non-autoregressive model family for our Sudoku comparison, with inference-time remasking [@wang2026remaskingdiscretediffusionmodels], confidence-ordered decoding [@kim2025trainworstplanbest], and PRISM-style self-correction [@kim2025selfcorrection]. A close stability-verification neighbor, @ruan2026reasoningmanifold use Bidirectional Manifold Consistency as a stability-based self-verifier for diffusion LMs. FRMs instantiate the same stability-as-correctness theme on flow-LM fixed points, then reuse the geometry to mine and localize wrong cells for `\flowdpo{}`{=latex}. #### Test-time scaling and verification. Our test-time rule is best-of-$N$, but the score is the candidate's fixed-point stability under the model's own re-noise-and-resolve dynamics. This differs from vote-based self-consistency [@wang2023selfconsistency], search over thoughts [@yao2023tree], and learned outcome or process verifiers [@cobbe2021gsm8k; @lightman2023letsverify]. Self-certainty [@kang2025selfcertainty] is the closest autoregressive verifier-free best-of-$N$ scorer here, but it reads distributional confidence; ours asks whether a completed state returns to itself after perturbation. Thus the scorer is not a popularity vote, auxiliary head, or label-trained verifier, and it stays useful where plurality collapses with difficulty (Section `\ref{sec:exp-verify}`{=latex}). #### Preference optimization on self-generated negatives. The training side uses direct preference optimization [@rafailov2024directpreferenceoptimizationlanguage] on self-generated negatives, consistent with evidence that suboptimal on-policy preference data can be especially useful [@tajwar2024preference]. The wrong-cell restriction is a fine-grained counterpart to step- or token-level preference localization [@lai2024stepdpo; @zeng2024tdpo]: it acts only where a spurious fixed point differs from gold. In our setting, the flow language model gives a closed-form, train-matched denoising-score contrast because both gold and wrong completions are scored under the same corrupted state (Eq. `\eqref{eq:loglik}`{=latex}). Related diffusion-preference objectives instead instantiate preference learning through diffusion-specific likelihood or ELBO-style estimators [@wallace2023diffusiondpo; @zhu2025vrpo; @borso2025d2dpo]; we leave a matched surrogate-versus-denoising-score ablation to future work. # Discussion {#sec:discussion} *Flow Reasoning Models* run a discrete flow model as a self-conditioned iteration on checkable reasoning tasks, where correct answers behave as stable fixed points that the dynamics recognize far better than the sampler reaches. We *drive* that structure to generate, *read* it to verify, and *reshape* it to train, on one set of weights. The uses are complementary but not interchangeable: reading rides a coverage ceiling the base sampler sets, while reshaping reconcentrates mass onto reachable solutions without itself guaranteeing coverage. Pushing coverage, through better proposers or learned exploration, is the part of the structure we do not yet drive. The view applies cleanly only where correctness is a checkable stable state; open-ended generation, settings without a clear fixed point, and larger scales are untested. Self-conditioning and `\flowdpo{}`{=latex} each train stably on their own, but their combination is harder to optimize and shows high seed-to-seed variance, so our strongest combined numbers are reported under seed selection; stabilizing the joint recipe remains future work. More broadly, evaluation for non-autoregressive structured reasoning is still immature at these scales, and the field would benefit from agreed small-scale benchmarks that isolate generation, verification, and training. ```{=latex} \bibliographystyle{template/iclr2026_conference} ``` ```{=latex} \appendix ``` # Self-conditioning: training, representation, and stability {#app:selfcond-details} This appendix collects the self-conditioning details deferred from Section `\ref{sec:method-selfcond}`{=latex}. #### The channel and its training. Self-conditioning adds a single *zero-initialized* input channel that carries the model's own previous-pass raw logits $s=\ell_{\text{prev}}$, embedded and added to the state before the backbone; because it is zero-initialized, an untrained model is byte-identical to the stock denoiser ($s{=}0$ recovers $D_\theta(x_t,t,0)$). We fit it with a stochastic two-pass scheme on top of the unchanged conditional cross-entropy objective (Eq. `\eqref{eq:ce}`{=latex}): with some probability the loss-bearing forward pass is conditioned on a first pass' logits taken under a stop-gradient, exactly as in @chen2023analogbitsgeneratingdiscrete; otherwise $s{=}0$. No labels, negatives, or test-time verifier are involved. #### Representation matters. We feed back *raw logits*, not normalized probabilities or a tanh-capped log-prob, so that inference matches the representation the channel was trained on; a train/inference mismatch here is destabilizing in practice (an earlier apparent bistability of the self-conditioned solve traced entirely to feeding tanh-capped log-probs at inference while training on raw logits, correcting it makes the $\sim$$0.98$ Sudoku result reliable across seeds). # Test-time scaling: coverage, selection, and sampling-step baselines {#app:tts} `\label{sec:exp-rejection}`{=latex} ```{=latex} \tablestyle{6pt}{1.2} ``` ::: {#tab:tts-scaling-shah} Sudoku (Shah) ------------------------------- ------------------------------------------ ---------- ---------- ----------- ----------- 2-6 `\footnotesize `{=latex}Number of rounds Method $1$ $4$ $16$ $64$ $256$ Base Flow Model 36.1 73.2 97.9 99.6 99.8 \+ Self Conditioning 99.8 100.0 100.0 100.0 100.0 **+ Self Conditioning / DPO** **99.1** **99.6** **99.9** **100.0** **100.0** : **Solve rate (%) on in-distribution [Sudoku]{.smallcaps} (Shah) as test-time compute scales** (the in-distribution counterpart of Table `\ref{tab:tts-scaling}`{=latex}). Columns give the number of *rounds*; each round proposes one candidate (a $128$-step inner loop) and verifies it with the renoise-CE self-verifier, keeping the best accepted answer. Self-conditioning alone is essentially sufficient ($99.8\%$ in one round), and `\flowdpo{}`{=latex} neither helps nor erodes it in distribution. ::: The two readings of the fixed point, sequential verify-and-restart (Algorithm `\ref{alg:self-refine}`{=latex}) and parallel best-of-$N$ selection (Eq. `\eqref{eq:selection}`{=latex}), both turn the stability signal into test-time scaling with no external feedback. We decompose the solve rate of a pool of $N$ proposals, $$\text{solve}(N) \;=\; \underbrace{\text{coverage}(N)}_{\text{a correct proposal exists}}\;\times\; \underbrace{\text{selection}\mid\text{coverage}(N)}_{\text{the verifier picks it}},$$ and measure each on a shared pool-size axis. Selection is settled (Section `\ref{sec:exp-verify}`{=latex}; top-$1$ accuracy $\approx 1.0$ even at the large pools the hardest split forces, Table `\ref{tab:selection}`{=latex}), so the residual is coverage. #### Coverage saturates, cheaply in distribution and expensively out of it. Oracle pass@$N$ rises with more proposals and saturates to $\sim$$100\%$; reading $N^{*}$ as the pool size reaching $95\%$ of that ceiling on the hardest tier, the correct solution is almost always *reachable* once the pool is large enough, cheaply in distribution (Shah $N^{*}{=}16$) and, on [Sudoku-Extreme]{.smallcaps}, only at a much larger pool ($N^{*}{\approx}\nstarExtremeBase$). The generator's reachable set transfers; the OOD split is harder only in *how many* proposals coverage costs, not in whether the answer is reachable at all. # Parallel selection {#app:propose-verify} ```{=latex} \fi ``` #### Parallel selection (the passive special case). From a pool of $N$ i.i.d. proposals $\{y^{(1)},\dots,y^{(N)}\}$, reading stability across independent attempts recovers a best-of-$N$ selector that keeps the most stable, $$y^{\star} = \arg\min_{i}\, S_{t,k}\big(y^{(i)}\big), \label{eq:selection}$$ with no external verifier and no ground truth, the purely *passive* use of the fixed point, where we never drive or reshape it, only rank candidates by how stable each one is. Scaling the pool raises solve rate toward the coverage ceiling; the sequential verify-and-restart of Algorithm `\ref{alg:self-refine}`{=latex} reaches the same ceiling more cheaply by spending its budget on a refined proposer (Appendix `\ref{app:tts}`{=latex}). # Renoise-CE selection after `\flowdpo{}`{=latex} {#app:selection-dpo} ```{=latex} \tablestyle{6pt}{1.15} ``` ```{=latex} \begin{subtable}{0.49\linewidth} \begin{tabular}{l YYY} \toprule & Top-1 Acc at Coverage $\uparrow$ \\ \cmidrule(lr){2-4} Metric & $90\%$ & $95\%$ & $99\%$ \\ \midrule Plurality vote & 0.81 & 0.85 & 0.97 \\ \ours \textbf{Renoise-CE} & \textbf{1.00} & \textbf{1.00} & \textbf{1.00} \\ \bottomrule \end{tabular} \caption{Sudoku (Shah et al.)} \end{subtable} ``` ```{=latex} \hfill ``` ```{=latex} \begin{subtable}{0.49\linewidth} \begin{tabular}{l YYY} \toprule & Top-1 Acc at Coverage $\uparrow$ \\ \cmidrule(lr){2-4} Metric & $90\%$ & $95\%$ & $99\%$ \\ \midrule Plurality vote & 0.69 & 0.76 & 0.84 \\ \ours \textbf{Renoise-CE} & \textbf{0.99} & \textbf{1.00} & \textbf{0.98} \\ \bottomrule \end{tabular} \caption{Sudoku-Extreme (OOD)} \end{subtable} ``` Table `\ref{tab:selection}`{=latex} shows the vote-versus-renoise-CE contrast for the *base* model. Table `\ref{tab:selection-dpo}`{=latex} repeats the measurement for the `\flowdpo{}`{=latex} model, each evaluated at its *own* (smaller) saturation pool $N^{*}$: the same contrast holds, in that the renoise-CE selects the correct grid essentially always, while the plurality vote lags, so the verifier's selection ability is not an artifact of the base model and survives preference training. Note `\flowdpo{}`{=latex}'s top-$1$ stays at $\sim$$1.0$ *despite* its smaller pool, so the proposer-efficiency gain of Table `\ref{tab:saturation}`{=latex} comes at no cost in selection accuracy. ```{=latex} \tablestyle{6pt}{1.15} ``` ```{=latex} \begin{subtable}{0.49\linewidth} \begin{tabular}{l YYY} \toprule & Top-1 Acc at Coverage $\uparrow$ \\ \cmidrule(lr){2-4} Metric & $90\%$ & $95\%$ & $99\%$ \\ \midrule Plurality vote & 0.89 & 0.93 & 1.00 \\ \ours \textbf{Renoise-CE} & \textbf{1.00} & \textbf{1.00} & \textbf{1.00} \\ \bottomrule \end{tabular} \caption{Sudoku (Shah et al.)} \end{subtable} ``` ```{=latex} \hfill ``` ```{=latex} \begin{subtable}{0.49\linewidth} \begin{tabular}{l YYY} \toprule & Top-1 Acc at Coverage $\uparrow$ \\ \cmidrule(lr){2-4} Metric & $90\%$ & $95\%$ & $99\%$ \\ \midrule Plurality vote & 0.88 & 0.91 & 0.95 \\ \ours \textbf{Renoise-CE} & \textbf{1.00} & \textbf{1.00} & \textbf{1.00} \\ \bottomrule \end{tabular} \caption{Sudoku-Extreme (OOD)} \end{subtable} ``` ```{=latex} \tablestyle{6pt}{1.15} ``` ```{=latex} \begin{subtable}{0.49\linewidth} \begin{tabular}{l YYY} \toprule & Pool Size $N^{*}$ at Coverage \\ \cmidrule(lr){2-4} Proposer & $90\%$ & $95\%$ & $99\%$ \\ \midrule Base & 11 & 16 & 27 \\ \ours \flowdpo{} & \textbf{6} & \textbf{8} & \textbf{22} \\ \bottomrule \end{tabular} \caption{Sudoku (Shah et al.)} \end{subtable} ``` ```{=latex} \hfill ``` ```{=latex} \begin{subtable}{0.49\linewidth} \begin{tabular}{l YYY} \toprule & Pool Size $N^{*}$ at Coverage \\ \cmidrule(lr){2-4} Proposer & $90\%$ & $95\%$ & $99\%$ \\ \midrule Base & 563 & 874 & 1370 \\ \ours \flowdpo{} & \textbf{434} & \textbf{593} & \textbf{1175} \\ \bottomrule \end{tabular} \caption{Sudoku-Extreme (OOD)} \end{subtable} ``` # Flow language model details and an optional few-step map {#app:flowmap} This appendix expands Section `\ref{sec:method-selfcond}`{=latex} with the noise schedule and the simplex-valued denoiser, and describes an optional few-step distilled map (not used by any reported result). We follow the flow / flow-map formulation of @boffi2024flowmapmatching [@lee2026flowmaplanguagemodels] and its discrete-domain recasting [@potaptchik2026discreteflowmaps; @albergo2026itomaps]. #### Schedule. We train on the linear interpolant of Eq. `\eqref{eq:path}`{=latex}, $x_t=(1-t)\varepsilon+t\,x_1$, with $t$ sampled on a reparametrized schedule: rather than drawing $t$ uniformly we draw a signal-to-noise level and map it through precomputed $\alpha$--$\gamma$ lookup tables, which concentrates training time where the denoiser is most sensitive. This is a standard variance-reduction choice and does not affect any quantity defined in the body. #### The denoiser is the simplex posterior. The Bayes-optimal denoiser is the conditional expectation of the clean data, $D_t(x)=\mathbb{E}[x_1\mid x_t{=}x]$. Because the endpoint corresponds to a discrete token at each position, this expectation is induced by the posterior distribution over the vocabulary and therefore lies on the simplex $\Delta^{|\mathcal{V}|-1}$ per position [@lee2026flowmaplanguagemodels]. Parameterizing it as a tokenwise softmax and matching it with cross-entropy (Eq. `\eqref{eq:ce}`{=latex}) trains in the simplex geometry of the categorical posterior; matching a Euclidean velocity instead is the mismatch @potaptchik2026discreteflowmaps identify and resolve by grounding the framework on the simplex. The velocity for the linear path is recovered in closed form from the denoiser, $v_t(x)=(D_t(x)-x)/(1-t)$ pushed through the embedding, so one network call yields the token distribution, the velocity, and the argmax readout. #### Optional few-step map (progressive semigroup distillation). The reported results deploy and probe the flow model itself, sampled in a handful of Euler steps. Where an even cheaper few-step deployment is wanted, the model can be distilled into a flow map $f_\theta(x_{t_a},t_a,t_c)$ satisfying the boundary, tangent, and semigroup conditions, via *progressive semigroup distillation* (PSD): for $t_a