Looped Transformers And Test-Time Memory

Summary

This topic tracks the post-Universal Transformer branch where models reuse computation over depth, retrieve from prior layer state, carry explicit segment memory, maintain explicit memory at test time, or combine these mechanisms. The common question is not just “can a model think longer?” It is which state is refined, retrieved, or carried forward by extra computation: token representations, depth key/value memories, learned memory slots, associative key-value memory, recurrent segment state, or recursive puzzle state.

For the wiki’s time-series and world-model agenda, this branch is architecture background. These sources should be cited as upstream evidence about dynamic compute, recurrent depth, test-time memory, or matched-budget evaluation, not as direct proof for numeric telemetry or action-conditioned world models.

flowchart LR
  UT[Universal Transformer]
  Depth[Looped and recurrent-depth Transformers]
  Memory[Test-time memory]
  Recursive[Small recursive reasoning models]
  UT --> Depth
  UT --> Recursive
  UT --> Memory
  Depth --> MoDA[MoDA / depth retrieval]
  Memory --> RMT[RMT / segment memory tokens]
  RMT --> ARMT[ARMT / associative memory]
  RMT --> RATE[RATE / action trajectories]
  Depth --> Huginn[Huginn / latent reasoning]
  Depth --> Illusion[Illusion of Superposition / latent CoT probes]
  Depth --> LTF[LTF / GFlowNet latent trajectories]
  Depth --> LoopWM[LoopWM / world-model loops]
  Depth --> LT2[LT2 / linear-time looped mixers]
  Depth --> Sleep[LLM Sleep / consolidation loops]
  Depth --> Hyperloop[Hyperloop / loop-level hyper-connections]
  Depth --> ELT[ELT / visual generation]
  Depth --> ThinkingPixel[Thinking Pixel / sparse diffusion latents]
  Depth --> LoopFormer[LoopFormer / elastic depth]
  Depth --> FPRM[FPRM / fixed-point halting]
  Depth --> FRM[FRM / perturb-and-resolve stability]
  Depth --> Parcae[Parcae / stable loops]
  Depth --> Sparse[Sparse looped MoE]
  Depth --> Samplers[parallel recurrent-depth samplers]
  ResidualStreams[mHC / matrix residual streams] --> Hyperloop
  Memory --> Titans[Titans]
  Memory --> ATLAS[ATLAS]
  Memory --> MIRAS[MIRAS]
  Memory --> MesaNet[MesaNet]
  Memory --> ContextCompression[LCLM / CAT context compression]
  Memory --> HOLA[HOLA / bounded exact cache]
  Recursive --> HRM[HRM]
  Recursive --> TRM[TRM]
  Recursive --> GRAM[GRAM]
  Recursive --> PTRM[PTRM]
  Recursive --> URM[URM]
  Recursive --> UTM[UT Need Memory]

Main Branches

Recurrent Depth And Latent Reasoning

Universal Transformers provide the root interface: reuse a Transformer block across depth, optionally halt per position, and treat repeated state refinement as a recurrent computation. The modern looped-language-model branch revisits that idea with large-scale pretraining and test-time compute.

Scaling up Test-Time Compute with Latent Reasoning is the scale proof in this batch: a recurrent-depth language model can use extra loops at inference to improve reasoning without producing more chain-of-thought tokens. Reasoning with Latent Thoughts supplies the matching theory and small-scale evidence that loops can simulate hidden reasoning steps for iterative problems. The Illusion of Superposition adds the needed warning: soft tokens and fine-tuned latent thoughts can collapse or shortcut, so hidden computation should be audited with no-latent ablations and state probes before calling it parallel reasoning.

Latent Thought Flow adds the positive, objective-design counterpart to that warning: instead of assuming hidden CoT naturally preserves multiple paths, it trains a continuous GFlowNet sampler over variable-length latent trajectories with an answer-quality and compute-cost reward. It should be read alongside Illusion because it proposes entropy-weighted and reference-prior machinery for avoiding deterministic trajectory collapse, but it still needs no-latent probes and wall-clock serving checks before hidden trajectories count as reliable multi-path reasoning.

Looped World Models moves recurrent depth into the action-conditioned world-model setting. The key transfer is not another language-model loop, but a latent dynamics core that refines state under observations and actions, can exit early on easy transitions, and can defer decoding during multi-step rollouts. It should be compared with LeWorldModel, Dreamer-style RSSMs, and stable-worldmodel baselines under matched rollout latency and simulator-transfer checks.

LoopFormer makes loop count a budget-conditioned interface through shortcut consistency. FPRM adds a different stopping contract: loop until the hidden state approaches a fixed point, while pre-norm plus residual scaling keep the loop trainable at very large effective depth. It is especially important because it challenges the assumption that recursive puzzle models need HRM/TRM-style fast/slow hierarchy; for this wiki, the transferable claim is convergence-controlled recurrent depth, not a direct time-series result. Parcae focuses on stable looped-model scaling laws. Sparse Layers are Critical to Scaling Looped Language Models argues that dense looping under-scales, while Looped-MoE can recover diversity through different expert routing across passes. LT2 attacks the other main cost center: recurrent depth with full attention multiplies KV-cache and attention cost, so looped models may need linear, sparse, or hybrid mixers before long-context looping is a practical serving interface.

Flow Reasoning Models is an adjacent but distinct fixed-point branch. FRM is not a looped Transformer: it closes a self-conditioning loop across discrete-flow denoising steps, then re-noises and re-solves the completed assignment. Its main contribution to this topic is the warning that convergence can land on a stable wrong state, plus a perturbation-based candidate-ranking signal that should be compared against FPRM residual halting rather than conflated with it.

Probabilistic Tiny Recursive Model adds a training-free stochastic-width branch on top of deterministic TRM. Gaussian noise at every deep-recursion step increases proposal coverage, and the inherited supervised Q head selects among parallel trajectories. The Maze-Hard gap between pass@$K$ and best-Q@$K$ makes the main systems lesson explicit: parallel exploration, selector calibration, and total rollout cost are separate resources.

Is One Layer Enough? adds a static-tabular branch to the looped-depth map. Its nanoTabPFN_looped experiment reuses one Transformer block six times and matches a six-layer nanoTabPFN baseline in a small controlled setting, while a one-pass one-layer model underperforms. This is useful evidence that repeated state refinement can matter outside language, but it is not a time-series or world-model result and it does not settle scaling to larger TFMs.

ELT extends the recurrent-depth branch into visual generation. Its important contribution is not only parameter sharing, but loop-boundary supervision: Intra-Loop Self Distillation makes intermediate loops useful enough for any-time image and video generation. For the wiki, that is an adjacent dynamic-compute pattern to compare with LoopFormer-style budget conditioning and early exits.

The Thinking Pixel adds a different visual-generation branch: recursive sparse LoRA-adapter experts refine continuous visual diffusion tokens inside joint attention layers. It is closer to latent ponder steps plus sparse expert routing than to whole-block looped depth, and it lacks ELT-style loop-boundary supervision or a public artifact release. Treat it as adjacent evidence that diffusion latents can accept recurrent sparse compute, not as proof that extra visual thinking is always cost-effective.

Efficient Parallel Samplers for Recurrent-Depth Models shifts from architecture to inference: if extra recurrent depth is useful, generation still needs a sampler that does not turn every additional loop into fully serial latency. The Recurrent Transformer is a related but distinct design where each layer maintains layerwise recurrent memory rather than looping a whole block.

LT2 also bridges this topic to the efficient recurrent-state page: linear-attention looped variants carry compact recurrent mixer state, while sparse looped variants trade exact full-context attention for a growing receptive field. The caveat is that its evidence is language modeling, synthetic recall/state tracking, and H100 throughput; it does not yet test numeric telemetry, event streams, or action-conditioned trajectories.

Gated DeltaNet-2 is not a looped-depth model, but it is relevant to the linear-mixer side of LT2-style long-context looping. It changes how fixed-size linear-attention state is edited by separating key-side erasure from value-side writing. The connection for this page is memory interference under a tight state budget: a looped or recurrent model that saves attention cost still needs to show that its compact state is not overwritten by irrelevant context.

Gated DeltaNet, Oryx, Hybrid Associative Memories, and HOLA form the hybrid-memory boundary around this page. Gated DeltaNet supplies the scalar-gated fast-weight state, Oryx switches between exact attention and recurrent mixers across sequence spans, HAM grows the KV cache only for hard-to-predict tokens, and HOLA spends a fixed top-$w$ exact-memory budget on the largest committed recurrent-state updates. They are not recurrent-depth methods, but they belong in the same long-context and semiparametric test-time-memory map because compact state, selective exact memory, and mode routing are substitutable ways to spend memory and latency.

NextLat is not a looped-depth architecture, but it is relevant to test-time compute because its learned latent dynamics model can recursively draft hidden states for variable-length self-speculative decoding. The distinction matters: extra compute is spent inside a trained latent transition head for drafting, not in repeated refinement of the base Transformer block.

Language Models Need Sleep adds a hybrid memory-consolidation branch. The model loops during the sleep phase before cache eviction, updates SSM fast weights, and then answers later with a single wake-time forward pass. That makes it a useful contrast to Huginn-style prediction-time recurrent depth: extra compute is spent on forming persistent fast state, not on refining the answer token at prediction time.

DiffusionBlocks adds a training-side recurrent-depth bridge. In its Huginn-style experiment, the repeated-depth dynamics are treated as a denoising process so training can use a single forward pass rather than BPTT through many recurrent iterations. This is not a test-time-memory method, but it changes the cost model for training looped-depth systems.

Depth Retrieval Without Looping

MoDA is adjacent to recurrent depth but solves a different interface problem. It does not loop the same block or add halting depth. Instead, it gives later layers content-based access to previous layer key/value memories and fuses sequence retrieval with depth retrieval in one softmax. This makes “which earlier layer should I use?” an attention decision inside the forward pass.

For the wiki, MoDA is useful because it separates two questions that are often merged: spending more depth compute and communicating across depth. A model can have many layers but weak inter-layer access, or fewer/looped layers with a stronger depth-memory interface. The caution is that MoDA introduces a depth-KV cache, so matched-budget comparison must include memory bandwidth and serving latency, not only nominal FLOPs.

Matrix Residual Streams For Looped Depth

mHC is not a looped Transformer paper, but it adds an important residual-state mechanism to this topic. It widens the residual stream into parallel streams and constrains residual mixing through the Birkhoff-polytope/Sinkhorn route, making the “state carried across layers” richer than a single hidden vector while trying to preserve residual stability.

Hyperloop Transformers applies that idea at loop boundaries in a middle-cycle looped Transformer. The result is a useful connection between recurrent depth and state capacity: a shared block can be parameter-efficient, but it may need matrix-valued residual state so repeated passes do not collapse into overly similar representations.

For the wiki, this is adjacent dynamic-compute evidence rather than time-series evidence. The right comparison is against unique-depth layers, MoDA-style depth retrieval, explicit memory tokens, and wider ordinary hidden states under matched parameter memory, KV/cache traffic, latency, and expected FLOPs.

Segment-Level Recurrent Memory

Recurrent Memory Transformer is the root segment-memory source: a Transformer segment is wrapped with read/write memory tokens, and the updated memory block is passed to the next segment. Unlike recurrent-depth models, the recurrence is over sequence segments rather than repeated refinement of the same tokens.

Associative Recurrent Memory Transformer strengthens the memory-capacity branch by adding layerwise associative memory to RMT. Its BABILong and associative-retrieval evidence makes it useful for thinking about mutable key/value state, but it remains language/long-context evidence rather than direct time-series evidence.

Recurrent Action Transformer with Memory adapts the segment-memory interface to offline RL trajectories. It matters here because its Memory Retention Valve is an explicit attempt to avoid losing sparse decision-relevant information across many trajectory segments.

Test-Time Memory And Online Optimization

Titans anchors the memory branch by combining attention as short-term memory with a learned long-term neural memory updated at test time. ATLAS extends this direction with a higher-capacity memory module that optimizes memory from current and past tokens, while MIRAS reframes attention, Titans, and modern recurrent models as associative memory systems defined by an attentional-bias objective, retention gate, and memory learning rule.

MesaNet is the online-optimization variant: the model spends inference compute to solve local in-context regression objectives. Titans Revisited is the cautionary reimplementation source: neural memory can help, but the full Titans stack does not automatically dominate strong baselines.

Dragon Hatchling belongs here as a boundary case: it is not a memory add-on to a Transformer, but the architecture itself updates a large recurrent fast state during inference. Its useful transfer is the state contract, sparse positive update pattern, and interpretability claim; its current evidence is language/translation rather than multivariate time series or action-conditioned trajectories.

Scaling Test-Time Compute for Agentic Coding adds an external-memory analogue. It does not loop a Transformer block or train a neural memory module. Instead, it turns prior agent rollouts into structured summaries, then uses those summaries as bounded memory for selection and refinement. This belongs on the topic page as a boundary case: test-time memory can live outside the hidden state, but then summary fidelity, provenance, and omission errors become first-class risks.

TurboQuant is a serving-memory compression boundary case. It does not add neural memory, memory tokens, or looped depth, but it compresses KV-cache and vector-search state while preserving inner-product geometry. The vLLM implementation critique makes this a useful caution for memory methods: compressed state can increase capacity while reducing throughput or adding per-token latency if the serving path must dequantize before attention. For time-series models, the transfer question is whether cached attention, retrieval memory, or latent-state stores can be quantized without erasing rare regimes, cross-channel deviations, exogenous variables, or action history, and whether that quantized path beats FP8/BF16-style baselines in wall-clock serving.

Latent Context Language Models and Compress & Attend Transformers are neural context-compression boundary cases. They do not update memory at test time like Titans or loop depth like Huginn; instead, they change the retained representation of long context before or during decoding. LCLM compresses long prompt spans into soft latent tokens before decoder prefill. CAT keeps the current chunk raw and represents older chunks through compressed chunk states. For this topic, they belong in the same budget map because compressed context, memory tokens, KV cache, SSM state, and external summaries are competing ways to retain history under latency and memory constraints.

Overlap Map

This topic intentionally groups mechanisms that can all look like “more memory” or “more thinking” from a distance. The wiki boundary is the state contract:

Cluster	Shared Question	Important Difference	Route Detailed Comparisons Through
RMT, ARMT, RATE	Can explicit memory carried across chunks replace full attention over long history?	RMT/ARMT carry memory across sequence segments; RATE applies the interface to offline RL trajectories and remains a policy model.	Efficient Recurrent Sequence Models, World Models for RATE boundary cases.
Titans, ATLAS, MIRAS, MesaNet	Can a model update memory or solve an objective during inference?	Titans/ATLAS emphasize neural long-term memory, MIRAS reframes memory objectives and retention, MesaNet spends compute on local optimization.	This page plus Time-Series Benchmark Hygiene for update cost, chunking, and baseline sensitivity.
Dragon Hatchling	Can a sequence model make mutable fast state the core architecture state?	BDH’s memory is a large recurrent state with sparse positive activations, not an external memory module or looped-depth schedule.	Efficient Recurrent Sequence Models plus this page for state size, update cost, and interpretability probes.
MoDA	Can later layers retrieve useful prior layer state directly?	MoDA stores depth KV within a forward pass; it is inter-layer retrieval, not segment memory or test-time memory.	Intermediate-Layer Representations, this page, and Time-Series Scaling And Efficiency.
mHC, Hyperloop Transformers	Can a richer residual stream make repeated depth less parameter-constrained?	mHC is a residual-stream mechanism; Hyperloop applies it at loop boundaries. This is state capacity across depth, not memory carried across time.	This page and Time-Series Scaling And Efficiency.
ELT	Can loop-boundary outputs remain useful at variable inference budgets?	ELT applies looped depth inside visual generation steps and trains intermediate exits with ILSD. This is visual-generation evidence, not TSFM evidence.	This page, Time-Series Scaling And Efficiency, and Vision Foundation Models.
The Thinking Pixel	Can sparse recursive latent steps refine continuous diffusion tokens without rerunning a full backbone?	Recursion is inside diffusion attention adapters, not whole-block looping or loop-boundary exits; evidence is image generation and toy navigation with no public code/model release.	This page, Mixture Of Experts, and Vision Foundation Models.
The Illusion of Superposition	Do latent thoughts actually preserve multiple candidate continuations?	It is a probe/ablation source, not a new looped architecture; it distinguishes forced soft-token mixtures, fine-tuned Coconut, and from-scratch latent training.	This page, Training Dynamics, and Representation Collapse.
Latent Thought Flow	Can latent reasoning sample a distribution over correct and cheap hidden trajectories rather than one path?	LTF uses continuous GFlowNet training and entropy-weighted subtrajectory balance over variable-length latent trajectories; evidence is LLM reasoning tasks, not numeric time series.	This page, LLM Post-Training, Representation Collapse, and Time-Series Scaling And Efficiency.
Looped World Models	Can recurrent-depth Transformers become action-conditioned world models?	LoopWM loops a shared dynamics block over latent environment state, adds spectral retention and early exit, and introduces deferred decoding for action-conditioned latent rollout; evidence is an unreleased technical report rather than a public benchmark artifact.	This page, World Models, Latent-Space Predictive Learning, and Time-Series Scaling And Efficiency.
nanoTabPFN_looped	Can a static-tabular in-context learner reuse one block across depth?	The recurrence is over effective TFM depth for tabular prediction, not over temporal history or action-conditioned latent rollout.	This page, Tabular Foundation Models, and Inference Dynamics.
DiffusionBlocks	Can recurrent-depth training avoid BPTT by using local denoising objectives?	DiffusionBlocks trains a Huginn-style recurrent-depth model as a denoiser with single-pass training, while keeping recurrent-depth inference.	This page plus Training Dynamics and Time-Series Scaling And Efficiency.
FPRM	Can convergence of the hidden state itself decide how long a looped model should run?	FPRM uses fixed-point residuals as a halting signal and stabilizes pre-norm looped depth with residual scaling, but evidence is symbolic reasoning rather than numeric time series.	This page, Time-Series Scaling And Efficiency, and Hierarchical Modeling with a Fixed FLOPs Budget.
FRM	Can perturbation stability of a completed iterative state verify a candidate better than generation confidence does?	FRM re-noises and re-solves discrete-flow puzzle completions; the signal ranks correct candidates nearly perfectly on the reported pools, but stable wrong attractors and proposal coverage remain.	This page, Inference Dynamics, Diffusion Language Models, and Time-Series Scaling And Efficiency.
PTRM	Can a pretrained deterministic recursive model gain useful parallel width without retraining?	PTRM repeatedly perturbs TRM latent state and selects with the inherited correctness Q head; Maze-Hard shows that proposal coverage can grow much faster than usable selected accuracy.	This page, Time-Series Scaling And Efficiency, and the PTRM source for cost and selector calibration.
Mamba, Mamba-2, Mamba-3, Gated DeltaNet-2, ParaRNN, RWKV-TS	Can compact recurrent state replace quadratic attention for serving?	These models keep state mostly as hidden recurrent dynamics or fixed-size fast weights rather than explicit memory slots. Gated DeltaNet-2 adds an erase/write interference-control variant.	Efficient Recurrent Sequence Models, Time-Series Scaling And Efficiency.
Universal-Transformer descendants and recursive small models	Can repeated computation refine the same representation?	The recurrence is over depth/refinement, not necessarily over streaming time or persistent memory.	This page; compare against memory methods only under matched FLOPs, latency, and state size.
LLM Sleep	Can extra recurrent compute make evicted context useful after the KV cache is cleared?	Recurrence happens during memory consolidation and updates SSM fast weights, not during every wake-time prediction.	This page and Efficient Recurrent Sequence Models.
LCLM, CAT	Can older or longer context be replaced by compressed latent representations before the decoder attends to it?	They compress context representation rather than adding mutable memory or repeated depth; preservation of rare details becomes the core risk.	This page, Latent Tokenization, and Time-Series Scaling And Efficiency.
Oryx, HAM, HOLA	Can exact attention be allocated only to spans that need it while recurrence handles the rest?	Oryx switches mixer mode across sequence chunks; HAM selectively grows a KV scratchpad; HOLA keeps a fixed top-$w$ exact cache by update magnitude and sharpens its cache read. All are language evidence and must be compared against sparse attention, context compression, and recurrent state under matched serving budgets.	Extra-Long Context For Time Series, Efficient Recurrent Sequence Models, and Time-Series Benchmark Hygiene.

Recursive Reasoning Branch

Hierarchical Reasoning Model, Tiny Recursive Model, FPRM, Flow Reasoning Models, Generative Recursive Reasoning, Probabilistic Tiny Recursive Model, Universal Reasoning Model, and Universal Transformers Need Memory form a small-model iterative or recursive reasoning cluster. They matter here because they test repeated state refinement under tight parameter budgets. FPRM specifically reframes the hierarchy question: fast/slow loops may be a workaround for post-norm signal propagation rather than the essential source of recursive-reasoning gains. FRM adds the convergence-versus-correctness distinction through perturb-and-resolve verification. GRAM trains stochastic latent transitions so recursive reasoning can scale in width as well as depth. PTRM obtains a simpler post-hoc width axis by perturbing a pretrained deterministic TRM, while showing that proposal coverage and selector calibration can diverge. All remain controlled puzzle or constraint-satisfaction evidence.

The lesson for time-series modeling is not that puzzle solvers transfer directly. The useful interface is the separation of latent state, update depth, supervision, memory slots, and halting or recursion schedule. Universal Transformers Need Memory is especially relevant because it makes the depth-state tradeoff explicit: extra recurrent passes and explicit memory tokens can substitute for one another up to a point.

Hyperloop Transformers adds the parallel residual-stream counterpart to that memory-token story. It does not prove a time-series or reasoning model, but it suggests that looped-depth models may need an explicit state-capacity budget even when the state is internal to the residual stream rather than exposed as memory tokens.

Time-Series Reading Frame

For time-series and digital-world agents, the key comparison is:

Mechanism	State being refined	Candidate TSFM use	Caution
Looped depth	token or patch representations	spend more compute on hard windows, rare regimes, or candidate futures	must beat unique-depth baselines under matched FLOPs, latency, and memory
Fixed-point looped depth	latent state plus residual-to-fixed-point statistic	halt when a window, transition, or candidate latent update stabilizes	residual convergence is not automatically calibrated uncertainty and may still shortcut under noisy or partially observed dynamics
Perturb-and-resolve stability	completed candidate state plus return-to-candidate score	rank or restart candidate future states after controlled perturbations	stability is not correctness under stochastic or multi-modal futures and may reject rare valid regimes
Stochastic recursive width	parallel perturbed latent-state trajectories plus selector score	spend parallel compute on several candidate futures or state updates	proposal coverage can improve without selector calibration; total compute, batching, and rare-regime selection must be measured
Looped world-model latent depth	action-conditioned latent environment state	spend more compute on hard transitions while rolling candidate action sequences forward in latent space	needs released artifacts, no-loop/no-deferred-decoding ablations, and numeric or operational action-conditioned benchmarks
Latent CoT / soft thoughts	hidden reasoning states or soft token mixtures	keep multiple candidate futures or regimes alive before committing	Illusion of Superposition shows collapse and shortcut risks; require no-latent ablations and state probes
Reward-proportional latent trajectories	stochastic continuous hidden thought states	keep multiple candidate futures or regimes alive before committing	no time-series evidence yet; hidden-state probes, reward design, and wall-clock serving cost dominate the claim
Sparse latent adapter recursion	continuous latent tokens inside a generator	route extra latent updates to hard spans, channels, regimes, or candidate futures	no numeric time-series evidence yet; realized latency, halting, and artifact release dominate the claim
Linear/sparse looped mixers	token representations plus compact mixer state or sparse KV slices	make extra loop iterations affordable for long observation histories	language throughput and synthetic state tracking do not prove dense numeric state retention
Depth-KV retrieval	previous layer key/value memories	retrieve intermediate state without manually choosing a layer	cache growth and memory bandwidth may dominate if depth slots are unbounded
Matrix residual streams	parallel residual streams across depth or loop boundaries	add state capacity to repeated-depth models without doubling unique layer parameters	memory access, fused kernels, and state-capacity accounting become part of the serving contract
Segment-level recurrent memory	learned memory tokens or associative memory carried between segments	retain state across long windows without attending over all prior observations	sequential segment processing, memory overwrite, and BPTT stability become central
Test-time memory	persistent learned memory state	retain long context, cross-variate history, and system context	update cost and memory capacity become serving constraints
Core fast-state architecture	large mutable recurrent state	keep streaming latent state inside the model rather than external memory	current evidence is language/translation; numeric streams and action channels are untested
Sleep-time consolidation	SSM fast-weight state before eviction	refresh latent state at finite-window boundaries in infinite streams	scheduling, training stability, and matched wall-clock latency dominate the claim
Block-wise denoising training	loop or depth update treated as denoising step	reduce BPTT or full-depth training memory for recurrent-depth models	local objectives must preserve global state and pretrained conversion is unproven
Structured rollout summaries	external compressed experience	reuse prior action/observation evidence without replaying raw traces	summary omission can erase action-relevant state
KV-cache and retrieval-memory quantization	compressed cached vectors	reduce memory footprint and indexing cost for retained sequence or retrieval state	inner-product fidelity may protect recall while still losing rare or action-relevant state; dequantization can erase serving gains
Online optimization	in-context objective solution	adapt to local regimes without full fine-tuning	extra inference FLOPs can erase efficiency gains
Recursive small models	compact latent puzzle state	study minimal recurrence and supervision structure	puzzle evidence is not telemetry evidence
Early exits	loop-boundary outputs	fast path for easy windows, uncertainty signal for hard windows	convergence is a diagnostic, not automatically calibrated uncertainty

Relation To Existing Topics

• Time-Series Scaling And Efficiency should cite this branch when discussing dynamic compute, looped depth, early exits, and memory/latency tradeoffs.
• Efficient Recurrent Sequence Models covers compact recurrent state, segment-level recurrent memory, and parallel training. This page covers how that branch relates to repeated Transformer depth and explicit test-time memory.
• Foundation Time-Series Model Research Agenda should treat this branch as adjacent dynamic-compute evidence unless a paper directly tests multivariate time-series latent state or action-conditioned trajectories.
• Mixture Of Experts becomes relevant when looped models use sparse experts to recover expressivity across repeated passes.
• Inference Dynamics covers the layer-probe and decoder-alignment side of repeated computation.

Open Questions

• When does extra loop compute outperform a wider or deeper unique-weight model at the same expected training FLOPs, serving latency, and memory footprint?
• What should a time-series memory store: recent values, channel relationships, latent regimes, exogenous context, interventions, or candidate futures?
• Should memory in action-conditioned time series be ordinary hidden state, explicit memory tokens, associative key/value memory, or a hybrid with a retention valve?
• Can loop convergence, memory-update magnitude, or halting depth become calibrated uncertainty signals for time-series systems?
• Can fixed-point residuals distinguish easy windows from shortcut convergence, especially when observations are noisy, partially observed, or action-conditioned?
• When does perturb-and-resolve stability add information beyond a fixed-point residual, predictive likelihood, ensemble disagreement, or an external constraint checker?
• When does stochastic recursive width beat ordinary ensembling or learned stochastic dynamics, and can an adaptive selector stop adding rollouts when marginal coverage saturates?
• Does test-time memory preserve rare regimes and cross-channel deviations, or mostly improve recall-style tasks?
• Can KV-cache or retrieval-memory quantization preserve rare regimes and action-relevant latent state while improving wall-clock serving, or does inner-product fidelity mostly protect recall-style tasks?
• Which interface is easier to serve in always-on systems: compact recurrent state, explicit memory slots, cached attention, or elastic looped depth?
• Which recursive-reasoning mechanisms transfer from discrete puzzle grids to numeric trajectories, event streams, or action-conditioned world models, and which only work because the puzzle state is small, discrete, and fully observed?
• Are explicit memory tokens, matrix residual streams, depth-KV retrieval, wider hidden states, and unique-depth layers additive or substitutable ways to increase state capacity in recurrent-depth models under matched latency, memory, and expected-FLOPs budgets?
• Can ILSD-style loop-boundary supervision make recurrent-depth time-series models genuinely useful at multiple inference budgets, or does it only stabilize visual generation exits?
• Can DiffusionBlocks-style denoising objectives train recurrent-depth time-series models without erasing long-range state that BPTT would otherwise expose?
• Can NextLat-style latent-dynamics drafting use acceptance statistics, hidden-state uncertainty, or rollout confidence to choose draft length online rather than relying on a fixed draft-length sweep?
• Can span-level mixer routing or selective KV-cache growth beat learned context compression and sparse attention after routing overhead, cache layout, and rare-state preservation are counted?