Alex Open Research Wiki

LeNEPA

3 min read

LeNEPA

Summary

LeNEPA is the no-augmentation next-latent-token prediction method introduced by LeNEPA: No-Augmentation Next-Latent Prediction for Time-Series Representation Learning. It trains a causal time-series encoder to predict the next patch embedding, uses temporal SIGReg for anti-collapse stabilization, and discards the projection head at frozen-feature evaluation time.

The method is the published MILETS 2026 realization of the earlier local LeNEPA idea: combine NEPA-style next-embedding prediction with LeJEPA/SIGReg-style distribution control and test whether this reduces domain-specific augmentation/view engineering.

Method Contract

  • Input surface: regular sampled time-series windows in the published experiments.
  • Tokenizer: Conv1d patch embedding before a causal ViT backbone.
  • Objective: next-latent-token MSE in a shared projected space.
  • Anti-collapse mechanism: temporal SIGReg over projected token sets inside each sample.
  • No stop-gradient target: unlike vanilla NEPA, the target side is not detached in the main LeNEPA objective.
  • Evaluation interface: frozen probes over intermediate backbone layers; the projector is training-only.
  • Current evidence: PTB-XL, Aionoscope Diag, and a CauKer-to-UCR frozen-encoder check.

Official Artifacts

The repository includes code, configs, experiment manifests, result tables, and paper assets. It intentionally does not vendor raw datasets, checkpoints, W&B run directories, VM logs, Hydra multiruns, or local outputs.

Role In The Wiki

LeNEPA belongs at the intersection of Next-Embedding Prediction, Latent-Space Predictive Learning, Representation Collapse, and Time-Series Classification Foundation Models.

Its local importance is that it moves the earlier LeNEPA design note from a hypothesis into a source-backed time-series SSL result. It still remains a passive representation-learning method in the published paper, not an action-conditioned world model.

Relation To Foundation TSFM Agenda

Use the source-level agenda mapping in lenepa-2026 rather than duplicating verdict rows here.

At the entity level, LeNEPA is a method object for testing whether augmentation-free latent prediction plus SIGReg can produce reusable time-series representations under low-retuning constraints. The open TSFM question is whether the same interface can preserve dense numeric state, rare regimes, channel relationships, exogenous variables, and eventually typed action/control-input histories.

Caveats

  • Evidence is regular-sampled and passive; no explicit actions, control inputs, interventions, or counterfactual rollouts are modeled.
  • The UCR check uses a single pretraining seed and best checkpoint over a trajectory.
  • The Aionoscope Diag result is controlled synthetic diagnostic evidence, not direct real-world transfer.
  • Intermediate layers are important; using only the final layer can understate the representation.

Graph View

Backlinks