LeNEPA: No-Augmentation Next-Latent Prediction for Time-Series Representation Learning

Source

• Raw Markdown: paper_lenepa-2026.md
• PDF: paper_lenepa-2026.pdf
• Preprint: https://arxiv.org/abs/2607.00958
• DOI: https://doi.org/10.48550/arXiv.2607.00958
• Official code: https://github.com/langotime/lenepa-milets-2026

Status And Credibility

This is an arXiv v1 preprint submitted on 2026-07-01 by Alexander Chemeris, Ming Jin, and Randall Balestriero. The arXiv record states that the paper was accepted by the 12th Mining and Learning from Time Series workshop at KDD MILETS 2026. The official repository is public under the langotime GitHub organization and contains code, experiment manifests, result tables, and paper assets for the MILETS/KDD 2026 submission.

Core Claim

LeNEPA argues that a time-series self-supervised learning recipe can reduce dependence on handcrafted augmentations by predicting the next latent token directly and stabilizing that prediction with temporal SIGReg instead of stop-gradient or EMA teacher updates.

The key evidence is a fixed-recipe stress test: reuse each method-specific SSL recipe unchanged after changing the pretraining signal family. In that protocol, the ECG-tuned JEPA masking recipe is strong in-domain on PTB-XL but weakens when reused unchanged on Aionoscope Diag, while LeNEPA preserves useful frozen-probe gains on both datasets.

Method Contract

LeNEPA uses a causal ViT-style backbone over time-series patch embeddings. For patch embeddings $z_t$ and causal predictions ${\hat{z}}_t$, a shared projector $h_{\psi }$ maps both sides into a lightweight training-only space:

$${\mathcal{L}}_{\mathrm{pred}}=\frac{1}{B(T-1)}\sum _{b=1}^B\sum _{t=1}^{T-1}{\Vert h_{\psi }({\hat{z}}_{b,t})-h_{\psi }(z_{b,t+1})\Vert }_2^2.$$

Unlike vanilla NEPA, LeNEPA does not stop gradients through the target tokens. Stabilization comes from temporal SIGReg over projected token sets inside each sample, and the projector is discarded for frozen-feature evaluation.

flowchart LR
  X[regular time-series window] --> P[conv patch embedding]
  P --> C[causal ViT backbone]
  C --> Pred[predicted latent token]
  P --> Target[next patch embedding]
  Pred --> Proj[shared projector]
  Target --> Proj
  Proj --> Loss[next-latent MSE + temporal SIGReg]
  C --> Probe[frozen probe reads intermediate layers]

Key Contributions

• Introduces Latent Euclidean Next-Embedding Prediction Architecture as a no-augmentation next-latent-token objective for time-series representation learning.
• Replaces NEPA’s stop-gradient/EMA stabilization with temporal SIGReg applied in a disposable projected space.
• Defines a fixed-recipe pretraining-reuse protocol that separates recipe portability from fully retuned benchmark performance.
• Evaluates PTB-XL and Aionoscope Diag with frozen linear probes, retraining each method independently on each dataset while keeping method-specific recipes unchanged.
• Adds a separate CauKer-pretrained LeNEPA frozen-encoder check on UCR-128 with a Random-Forest classifier.

Evidence And Results

• PTB-XL / Diag fixed-recipe reuse: the ECG-tuned JEPA recipe remains strong in-domain but loses strength when reused unchanged on Diag, while LeNEPA keeps useful frozen-probe gains in both signal families.
• Early representation acquisition: the paper reports that LeNEPA reaches 80% of its final AUROC/AUPRC gain after 2—5k updates in the fixed-horizon protocol, while the faster JEPA readout reaches the same threshold after 5—10k updates.
• UCR-128 frozen-encoder check: a CauKer-pretrained LeNEPA variant reaches 77.65% mean UCR-128 Random-Forest accuracy in a single-seed, best-checkpoint run, within 1.16 points of Mantis and within 0.24 points of MOMENT under the cited protocol anchors.
• Layer sensitivity: intermediate layers, especially around a fixed L4 readout for the classification summaries, can outperform the final layer. This links LeNEPA to the broader wiki tension around best-layer versus final-layer representations.

Relation To Aionoscope

LeNEPA uses Diag, a synthetic diagnostic corpus generated with Aionoscope, as the non-ECG signal family in the fixed-recipe stress test. In the LeNEPA paper, Diag functions as controlled instrumentation: labels are used after pretraining to probe which categorical and dense process variables are linearly recoverable from frozen representations.

Limitations

• The paper tests recipe portability, not the best possible Diag-tuned JEPA recipe and not a universal foundation encoder.
• The main PTB-XL/Diag setting uses regular, fixed-length sequences and a Conv1d tokenizer; irregular time, event streams, high-dimensional native multivariate telemetry, and typed action/control-input histories remain untested.
• Aionoscope Diag is synthetic diagnostic evidence, not direct proof of real-world diagnostic transfer.
• Temporal SIGReg still has scale and layer-placement hyperparameters; the CauKer/UCR run uses a different temporal scale.
• Some comparisons use best checkpoints, oracle layer choices, or a single pretraining seed and should be treated as diagnostic evidence rather than a stable leaderboard claim.
• LeNEPA is a passive representation-learning method in this paper. It does not model actions, control inputs, interventions, or counterfactual rollout.

Foundation TSFM Relevance

Agenda slot	Verdict	Evidence	Missing pieces
Augmentation-free or dataset-aware self-supervision	partially closes	Direct time-series SSL evidence that no-augmentation next-latent prediction can remain useful when a recipe is reused across PTB-XL and Aionoscope Diag.	Needs broader multivariate, irregular, event-stream, and operational domains; compare against stronger general-purpose non-ECG JEPA recipes.
Anti-collapse regularization	partially closes	Temporal SIGReg stabilizes next-latent prediction without stop-gradient or EMA in the reported recipe.	Need collapse, rare-regime, dense-state, and long-tail audits beyond the tested datasets and probe stack.
Representation quality	partially closes	Frozen probes show useful intermediate-layer representations and external UCR-128 neighborhood-level evidence.	Need dense numeric preservation, high-channel state, action history, uncertainty, and real-task transfer probes.
Benchmark level	warning	The paper explicitly frames the protocol as fixed-recipe stress testing, not a fully tuned leaderboard.	Future comparisons should report tuning budget, layer-selection budget, seed variance, and benchmark leakage controls.
Control and counterfactuals	insufficient evidence	The method is passive and uses no action, control-input, or intervention channel.	Add typed actions or interventions and evaluate candidate-action rollout or counterfactual prediction.

Links Into The Wiki

• LeNEPA
• LeNEPA idea
• Next-Embedding Prediction
• Latent-Space Predictive Learning
• Latent-State Time-Series Modeling
• Distribution Priors In Self-Supervised Learning
• Intermediate-Layer Representations
• Representation Collapse
• Time-Series Classification Foundation Models
• Time-Series Benchmark Hygiene
• Foundation Time-Series Model Research Agenda

Open Questions

• Does temporal SIGReg remain stable when the tokenizer, sampling rate, channel count, or signal family changes more aggressively?
• Can LeNEPA preserve dense numeric state and rare event timing under nonlinear or token-level probes, not only pooled frozen probes?
• Which target family is best for future LeNEPA variants: external patch embeddings, own-hidden-state targets, contextual embeddings, or a hybrid?
• Can the no-augmentation recipe extend to irregular time series and event streams without reintroducing domain-specific preprocessing heuristics?
• What action-conditioned version of LeNEPA would support counterfactual rollout rather than only passive representation learning?