Inference Dynamics
Summary
Inference dynamics tracks how a model’s internal state changes during a forward pass or repeated inference computation. It is different from Training Dynamics: the question is not how optimization shapes weights over training, but how representations, predictions, calibration, memory, or latent state evolve while a trained model answers.
Current Evidence
Is One Layer Enough? is the current tabular anchor. It studies six tabular foundation models and finds staged inference over depth: latent mapping, feature engineering and labeling, prediction ensembling, and prediction calibration. The paper’s tuned-decoder results show that intermediate states can already contain predictive information before they are aligned with the original final decoder.
The source also provides a small looped-depth result: a single nanoTabPFN block reused six times can match a six-layer nanoTabPFN baseline in the reported setting, while a one-pass one-layer model underperforms. This supports repeated state refinement as a plausible tabular inference mechanism, but only in a small static-tabular setting.
Reading Frame
For this wiki, inference-dynamics claims should name:
- the state being tracked: residual stream, recurrent state, memory token, depth key/value state, decoder logits, prediction entropy, or task-specific probe state;
- the probe or intervention: tuned decoder, logit lens, probing classifier, layer skip, layer swap, layer repeat, early exit, halting statistic, or recurrent loop count;
- the data contract: static tabular context, time-series history, event stream, action-conditioned trajectory, text prompt, or another modality;
- the budget contract: unique parameters, effective depth, expected FLOPs, wall-clock latency, memory bandwidth, and whether early exits are actually served.
Time-Series Boundary
Static tabular inference dynamics are adjacent to time-series modeling, not direct evidence for it. A multivariate time-series model must preserve temporal order, channel identity, rare regimes, exogenous variables, events, and possibly actions or control inputs. A useful time-series inference-dynamics study would probe whether those variables survive across depth, recurrent updates, early exits, and decoder alignment.
Open Questions
- Can per-layer decoder alignment become a reliable early-exit interface, or does it leak assumptions from each model’s synthetic prior?
- When looped depth improves a static tabular model, does the gain survive matched latency, expected FLOPs, memory bandwidth, and larger-context scaling?
- Which time-series variables survive across depth or recurrent inference updates: temporal order, channel identity, rare regimes, events, exogenous variables, and control input history?