Skip a Layer or Loop It? Learning Program-of-Layers in LLMs

Source

• Raw Markdown: paper_polar-2026.md
• PDF: paper_polar-2026.pdf
• Preprint: arXiv:2606.06574v2
• Official ICML 2026 oral page: https://icml.cc/virtual/2026/oral/71114
• OpenReview forum: https://openreview.net/forum?id=pl10b6EQAN
• Official code: tianyi-lab/PoLar, audited at commit 30d0efde953a5d139e220f61a773c0eaf92d478f
• Gonzo ML discussion: Telegram post 5950 (exact root-post snapshot stored at papers/polar-2026/telegram-post-gonzo-ml-5950.md)
• Local artifact records: papers/polar-2026/official_artifacts_metadata.json, github_repository_metadata.json, and code_artifact_audit.json

Status And Credibility

The source is current, tier-1 conference work. arXiv records version v1 on 2026-06-04 and version v2 on 2026-08-08; v2 is the ingested revision. The official ICML 2026 program lists the paper as an oral, links the OpenReview forum, and names Ziyue Li, Yang Li, and Tianyi Zhou. The arXiv record lists University of Maryland and Mohamed bin Zayed University of Artificial Intelligence affiliations, a CC BY 4.0 license, and an explicit revision note: DART-Math examples were deduplicated before splitting and the affected in-distribution analyses were rerun.

Credibility is strong because of the ICML oral status, multi-backbone experiments, released implementation, and explicit v2 data-cleaning correction. Reproducibility is nevertheless partial: the repository releases the predictor and layer-path execution code but not the MCTS search pipeline, the generated merged_mcts_samples.json supervision, trained PoLar checkpoints, benchmark outputs, tests, CI, or a repository license.

Core Claim

A frozen pretrained Transformer can sometimes answer better when its existing layers are treated as callable functions and executed through an input-specific program instead of the standard one-pass order. PoLar learns a small controller that partitions depth into short contiguous segments and assigns each segment one operation: skip it, execute it once, or execute it twice.

This is adaptive compute over model depth, not adaptive processing over temporal history. It changes which pretrained layer functions run and how often; it does not change the base-model weights.

Mechanism

PoLar separates expensive offline discovery from cheap online routing.

flowchart LR
  MCTS["offline MCTS over layer paths"] --> Labels["valid program supervision"]
  X["input x"] --> Enc["frozen Qwen3-Embedding-0.6B"]
  Enc --> Pred["PoLar predictor"]
  Labels --> Pred
  Layers["learned queries for layer indices"] --> Pred
  Pred --> Seg["contiguous segments, max length 4"]
  Seg --> Ops["skip / keep / repeat once"]
  Ops --> Beam["small beam of execution programs"]
  Beam --> LLM["frozen base LLM executes selected paths"]
  LLM --> Answers["one answer per executed program"]

For a base model with depth $D$, PoLar predicts segment starts

$${\mathbf{z}}^{\mathrm{seg}}(x)\in \{0,1{\}}^D$$

and an operation label at each segment start,

$${\mathbf{z}}^{\mathrm{op}}(x)\in \{\text{skip},\text{keep},\text{repeat}{\}}^D.$$

A segment $[s_j,s_{j+1})$ is omitted, run once, or concatenated with one extra copy. The paper caps segment length at four and allows one additional execution because the offline MCTS traces were dominated by short, local segments and usually no more than one recurrence.

The predictor uses token representations from a frozen Qwen3-Embedding-0.6B, projects them to dimension 256, cross-attends learned layer-index queries to those tokens, applies a two-block Transformer over layer depth, and emits boundary and operation logits. It is trained with segmentation binary cross-entropy plus operation cross-entropy on MCTS-discovered valid programs. At inference, segmentation is thresholded and a small beam search ranks operation combinations.

What The Experiments Show

Evidence	Paper result	Calibrated reading
MCTS existence study	Across four frozen LLMs and five DART-Math difficulties, skip-only and loop-only searches find valid alternative programs, while the combined search finds many more.	This establishes that useful nonstandard paths exist under an answer-checked search budget; it is an oracle diagnostic, not the deployed predictor’s accuracy.
Shorter valid paths	71.9% of inputs already solved by the standard path and 34.0% of inputs corrected by another path admit a shorter valid program.	Standard depth can over-compute on many examples, but finding the short valid path offline uses correctness feedback.
LLaMA-3.2-3B pass@1 on DM-1	Base sampling 48.9%; PoLar 54.6%. Base greedy is 51.1%.	The paper-highlighted +5.7 points is versus its sampling row; versus greedy the same PoLar row is +3.5 points. Pass@1 is the cleanest evidence for a usable single predicted program.
LLaMA-3.2-3B macro pass@5	Base sampling 43.8%; PoLar 51.2%.	Positive proposal-set coverage, but it checks five program executions and counts success if any is correct. This is not single-program latency or selected-answer accuracy.
Qwen1.5-MoE-A2.7B latency	Base 373.45 ms; PoLar DM-1 311.41 ms with +5.7 points; PoLar DM-5 353.31 ms with +9.4 points. Reported controller overhead is 3.05 ms.	A useful one-model, two-difficulty wall-clock result. It does not establish latency gains for every backbone, every $k$, batch regime, or serving engine.
OOD pass@1	The Qwen1.5-MoE-A2.7B table reports gains over the standard forward pass on ASDiv, MAWPS, and all listed MMLU-Pro subjects.	Evidence that the learned depth policy is not only a same-split lookup, but still language/math and multiple-choice evidence rather than time-series transfer.

Pass@$k$ And Deployment Semantics

The paper defines pass@$k$ as success when at least one of the top-$k$ predicted execution programs produces a correct answer. The released evaluator does exactly that: it executes or cache-checks up to top_k_paths candidates and sets has_correct when any succeeds.

For $k>1$, this is an oracle-set metric. It measures proposal coverage after multiple base-model executions, but the release does not provide a deployment-time verifier that can identify which candidate answer is correct without ground truth. The useful operational quantities must therefore be reported separately:

1. pass@1 or top-program accuracy;
2. pass@$k$ proposal coverage;
3. selected accuracy under a real verifier or confidence rule;
4. total latency, FLOPs, memory, and base-model executions for all $k$ candidates.

This is the same proposal-versus-selector distinction that appears elsewhere in the wiki’s test-time-compute literature.

Social-Post Claim Audit

The Telegram post is a useful summary, but three phrases need tighter scope.

Root-post wording	Audit
”метод без дообучения базовой модели (training-free)“	The base LLM is frozen, but the PoLar predictor is trained with supervised labels from offline MCTS. “No base-model fine-tuning” is accurate; “training-free system” is too broad.
”легковесный предиктор на 2.1 млн параметров”	The paper’s learned predictor heads are about 2.11–2.12M parameters. The released pipeline also loads frozen Qwen3-Embedding-0.6B; the 2.1M figure is not the total auxiliary weight footprint.
”стабильно лучше по качеству а часто ещё и быстрее”	The reported pass@1 tables and one Qwen1.5 latency study support the direction. The speed claim should not be carried over to pass@$k>1$, which can execute several full candidate programs.
”пропускает … на простых … и повторяет … на сложных”	The paper finds that harder inputs more often require nontrivial skipping and/or recurrence, but the controller is not supervised with an explicit easy/hard label and does not guarantee that simple inputs only skip while hard inputs only repeat.

Paper-Code Audit

The official release implements the main predictor and custom layer-path execution, but it does not reproduce the full v2 paper protocol without modification.

DART-Math split drift

The v2 paper deduplicates each difficulty level to 565, 1,349, 1,579, 1,537, and 1,577 examples and uses approximately 62.5% / 12.5% / 25% train/validation/test splits, totaling 4,130 / 826 / 1,651 examples.

The audited code instead hard-codes the same offsets for every per-difficulty file:

• train: 0:1250;
• validation: 1250:1500;
• test: 1500:2000.

Those offsets cannot implement the published proportional v2 split and produce empty validation or test slices for difficulty files shorter than the corresponding boundary. A reproduction must reconstruct the deduplicated pools and add explicit split indices or update the slicing logic. The README was last committed before v2 and also retains a stale statement that the corresponding paper version was not yet public.

Artifact boundary

• The expected merged_mcts_samples.json files are absent.
• The offline MCTS generation code is absent.
• No trained PoLar checkpoint or tagged release is present.
• No test suite or CI configuration is present.
• GitHub detects no license and the audited tree has no LICENSE* file.
• Evaluation trusts cached final_valid_transitions by default; --no_trust_valid_cache reruns cache hits online, but still requires the unreleased supervision records.
• model.state_dict() includes the frozen embedding model submodule, so checkpoint and deployment accounting should separate the trainable 2.1M controller from the encoder weights it depends on.

Relevance To This Wiki

PoLar adds an important architecture pattern to Looped Transformers And Test-Time Memory: recurrent depth and layer skipping can be composed into an input-specific whole-program decision before the base model runs, rather than chosen locally layer by layer or fixed at training time.

For Time-Series Scaling And Efficiency and Hierarchical Modeling with a Fixed FLOPs Budget, the transferable mechanism is a depth controller over a frozen backbone. A time-series analogue could encode an observation window plus exogenous variables and, for a world model, actions or control inputs; then predict which contiguous depth segments to skip, keep, or repeat.

The transfer remains hypothetical. PoLar does not model temporal order, multivariate numeric state, irregular sampling, event streams, actions, interventions, uncertainty, multi-step latent rollout, or closed-loop control. Its frozen semantic text encoder would also need replacement or careful validation: dense numeric preservation and rare-event sensitivity cannot be assumed from language embeddings.

Foundation TSFM Relevance

Agenda slot	Verdict	Evidence	Missing pieces
Dynamic compute allocation	adjacent	Predicts input-specific skip/keep/repeat programs over frozen depth and reports pass@1 quality plus a limited wall-clock latency result.	No time-series data, no explicit expected-FLOPs constraint, no hard per-request budget, and no matched TSFM serving benchmark.
Streaming state and long context	insufficient evidence	The method changes computation depth for one prompt.	No retained state across windows, event-stream update, missingness handling, or unbounded-history test.
Representation quality	warning	Layer programs can change which internal transformations create the final answer.	No probes for dense numeric detail, rare regimes, cross-channel state, exogenous variables, or action history; routing uses a frozen text embedding model.
Control and counterfactuals	insufficient evidence	None in the evaluated interface.	Must condition on actions/control inputs and evaluate action-sensitive next-state or rollout predictions.
Benchmark hygiene	warning	v2 removes duplicated DART-Math questions and states the corrected split.	Released code uses stale fixed offsets and omits the MCTS supervision, search pipeline, checkpoints, tests, and selector evaluation.

Links Into The Wiki

• Program-of-Layers
• Looped Transformers And Test-Time Memory
• Time-Series Scaling And Efficiency
• Inference Dynamics
• Hierarchical Modeling with a Fixed FLOPs Budget
• Contradictions And Open Tensions
• Universal Transformers
• Looped World Models
• FPRM
• Probabilistic Tiny Recursive Model

Open Questions

• Can a verifier convert PoLar’s pass@$k$ proposal coverage into selected accuracy without ground-truth answer checks or erasing the latency gain?
• Does a controller trained on reconstructed v2 splits reproduce the reported pass@1 and latency results, and how much do they depend on trusting MCTS validity caches?
• What is the full deployment footprint and latency when the frozen Qwen3 embedding encoder, checkpoint storage, batching, and all candidate executions are counted?
• Does a compute-budget loss improve the quality-versus-latency frontier compared with PoLar’s correctness supervision plus path-length reweighting?
• Can segment-level depth routing preserve dense multivariate time-series state, rare events, and action effects better than early exit, per-layer routing, or an ordinary larger/smaller backbone under matched cost?