HEPA: A Self-Supervised Horizon-Conditioned Event Predictive Architecture for Time Series

Source

• Raw Markdown: paper_hepa-2026.md
• PDF: paper_hepa-2026.pdf
• Preprint: https://arxiv.org/abs/2605.11130v4
• DOI: https://doi.org/10.48550/arXiv.2605.11130
• Official project page: https://www.forgis.com/papers/hepa.html
• Official code: https://github.com/Forgis-Labs/HEPA
• FMSD @ ICML 2026 listing: https://icml-structured-fm-workshop.github.io/accepted-papers/
• OpenReview: https://openreview.net/forum?id=TdpqVBB8na

Status And Credibility

HEPA is an arXiv preprint first submitted on 2026-05-11 and revised to v4 on 2026-06-03. It was accepted as a Spotlight Oral at the 2nd ICML Workshop on Foundation Models for Structured Data (FMSD), not as an ICML main-conference paper. The official FMSD accepted-paper page independently lists the spotlight. The authors released code, reproducibility notes, tests, and result artifacts.

This is current and unusually close to the wiki’s core work: it is direct time-series JEPA evidence, not a vision-to-time-series analogy. Credibility is supported by the peer-reviewed workshop spotlight and public code, but the result remains a 2026 preprint/workshop paper without independent replication.

Core Claim

HEPA reframes heterogeneous rare-event tasks as one horizon-conditioned event-prediction interface:

$$P(\text{event within }\Delta t\mid x_{\le t}).$$

A causal Transformer learns predictive representations from unlabeled history. Instead of discarding the JEPA predictor after pretraining, HEPA freezes the encoder and fine-tunes the horizon-conditioned predictor plus a small event head. The head emits conditional hazards whose product forms a monotonic survival CDF.

Method Contract

flowchart LR
  X[history x up to t] --> E[causal Transformer encoder]
  E --> H[latent state h_t]
  H --> P[horizon-conditioned predictor]
  D[horizon delta t] --> P
  F[future interval during SSL only] --> T[shared target encoder]
  P --> L[latent L1 loss + SIGReg]
  T --> L
  P --> Z[conditional hazard head after pretraining]
  Z --> CDF[monotonic event CDF]

The encoder is a 2-layer, 4-head causal Transformer with width 256 and non-overlapping patches of size 16. A 2-layer MLP predictor receives the encoder state and prediction horizon. During self-supervised pretraining, it predicts a shared-encoder representation of the future interval. The objective combines latent L1 alignment with SIGReg rather than an EMA target encoder or stop-gradient branch.

After pretraining, the encoder is frozen. For each discrete horizon $\Delta t$, the predictor and event head emit a conditional hazard ${\lambda }_{\Delta t}(t)$. The probability of at least one event by that horizon is

$$p(t,\Delta t)=1-\prod _{j=1}^{\Delta t}\left(1-{\lambda }_j(t)\right).$$

This parameterization makes event probability non-decreasing in the horizon by construction.

What Is Actually Shared Across Domains

HEPA does not release one cross-domain pretrained foundation-model checkpoint. A separate encoder is pretrained on each dataset. What transfers across 14 benchmarks is the architecture, hyperparameter recipe, horizon-conditioned objective, predictor-fine-tuning interface, and h-AUROC evaluation protocol. The paper explicitly names cross-domain pretraining on FactoryNet as future work.

That distinction matters for the foundation-model claim: HEPA demonstrates a reusable recipe for latent event prediction, not yet a universal industrial representation.

Evidence And Results

• The v4 paper reports that HEPA wins 10 of 14 full-label benchmarks against matched frozen-encoder baselines while tuning 198k parameters, versus 2.16M for end-to-end training.
• On C-MAPSS-1, HEPA retains 92% of full-label h-AUROC using 2% of labels: 2 labeled engines out of 85.
• On the four C-MAPSS variants, full-label h-AUROC is 0.81, 0.57, 0.84, and 0.63 in the main table.
• The method wins 6 of 14 at 10% labels, so label efficiency is not universal across all event types.
• The clearest gains occur for extended precursor dynamics. The paper reports weaker or unstable few-label behavior on short-window anomaly datasets where the future target interval may carry little event information.
• The latent space learned on C-MAPSS-1 forms a smooth degradation manifold: PC1 captures 61% of variance and the median per-engine Spearman correlation with time-to-failure is reported as 0.97.
• The within-dataset checkpoint analysis supports a negative relationship between pretraining error and h-AUROC on C-MAPSS-3 and MBA; the SMAP correlation is weaker and not statistically significant at the reported sample size.

Claim And Artifact Drift

The official project webpage says HEPA wins 9 of 14, while arXiv v4 and the official code README say 10 of 14. This page follows the canonical v4 paper for the headline count.

The code README also states that its C-MAPSS reproduction values are higher than the arXiv paper because the released recipe uses a fixed fine-tuning epoch budget instead of validation-loss early stopping. Reproduction reports MUST distinguish paper-faithful numbers from the updated repository protocol.

h-AUROC As An Evaluation Contribution

HEPA treats the full output as a probability surface $p(t,\Delta t)$ and defines h-AUROC as a uniform mean of per-horizon AUROC values. This avoids pooling horizons with radically different event prevalence and supplies a common threshold-free metric across degradation, anomaly, and medical-event datasets.

The metric is useful but not sufficient by itself:

• uniform horizon weighting may not match operational costs;
• AUROC does not establish calibration;
• domain metrics such as RUL RMSE and point-adjusted F1 are lossy projections with different failure modes;
• one aggregate h-AUROC can hide poor short-horizon or rare-regime behavior.

Limitations

• No single cross-domain pretrained checkpoint; encoders are trained per dataset.
• Evidence is passive event prediction, not an action-conditioned world model.
• Channel-fusion tokenization loses to per-variate or channel-independent baselines on some sensor-localized events.
• HEPA is below the best baseline on four full-label datasets and wins only 6 of 14 at 10% labels.
• The theoretical information-retention bound depends on target sufficiency, Lipschitz posterior assumptions, bounded posterior probabilities, and prediction-error control; constants are not estimated empirically.
• The code and paper use different C-MAPSS fine-tuning stopping rules.
• The benchmark suite reuses public datasets and per-dataset preprocessing; it is not evidence of cross-machine zero-shot transfer.

Foundation TSFM Relevance

Agenda slot	Verdict	Evidence	Missing pieces
Latent-state prediction	partially closes	Direct time-series JEPA predicts future representations and exposes a horizon-conditioned latent event interface.	Need one reusable cross-domain checkpoint, streaming state updates, and dense-state probes beyond event labels.
Augmentation-free self-supervision	partially closes	Uses future-representation prediction with SIGReg rather than handcrafted positive-pair augmentations.	Needs cross-dataset pretraining and robustness to irregular time, event streams, missingness, and high-channel telemetry.
Anti-collapse regularization	partially closes	Direct time-series evidence for SIGReg with a shared target encoder and no EMA/stop-gradient target.	Need rare-regime, non-Gaussian, channel-localized, and long-tail audits.
Rare-event and label-efficient modeling	partially closes	92% retention at 2% labels on C-MAPSS-1 and consistent gains on extended-precursor tasks.	The advantage contracts on short-window anomalies and is not universal at 10% labels.
Benchmark level	partially closes	Unifies event tasks through a horizon-indexed probability surface and h-AUROC.	Add calibration, cost-weighted horizons, early-warning utility, false-alarm burden, and live-stream evaluation.
Control and counterfactuals	insufficient evidence	No action, control-input, intervention, or candidate-control rollout channel is modeled.	Combine with S-E-F-C-style control inputs and evaluate consequences under alternative controls.

Why It Matters For This Wiki

HEPA is a direct bridge between the local LeNEPA line and operational event prediction. LeNEPA asks whether next-latent self-supervision yields portable frozen representations; HEPA asks whether the predictor itself should be retained and specialized toward horizon-indexed events. The most important shared intuition is that the latent target and readout interface can separate learning dynamics from unlabeled streams from mapping those dynamics to scarce downstream labels.

The boundary is equally important: HEPA learns one dataset at a time and predicts passive event probability. FactoryNet supplies the missing multi-embodiment corpus and explicit commanded-intent/feedback schema; FactoryBench supplies a reasoning and decision benchmark. Together they form a coherent but not yet integrated industrial-TSFM stack.

Links Into The Wiki

• HEPA entity
• FactoryNet
• FactoryBench
• LeNEPA
• Joint Embedding Predictive Architecture
• Latent-Space Predictive Learning
• Latent-State Time-Series Modeling
• Time-Series Benchmark Hygiene
• Foundation Time-Series Model Research Agenda

Open Questions

• Does predictor fine-tuning still win when one encoder is pretrained across FactoryNet embodiments rather than separately per benchmark?
• Can a horizon-conditioned latent predictor jointly output event risk, calibrated future-state uncertainty, and dense numeric rollouts?
• Does S-E-F-C channel structure fix HEPA’s weakness on sensor-localized events better than sensor-as-token encoding alone?
• How should h-AUROC be combined with calibration, intervention utility, and false-alarm cost for deployment?
• Can the same predictor be conditioned on candidate control inputs so it estimates how intervention choices move event-risk mass across horizons?