Concept–Residual Expansion For Time Series

Idea in one sentence: freeze a foundation time-series encoder, expose interpretable temporal factors plus a complementary residual view, and test whether a small readout can stop relying on source-specific shortcuts under real temporal, site, device, and regime shifts.

Status: draft research idea derived from CRX and the local foundation-TSFM agenda; no temporal implementation or experiment has been run yet.

Collaboration

If this direction resonates with you, I would be happy to talk with like-minded people, collaborate on research, and work on use-cases together.

Ideas are not the bottleneck. Hands are. Time-series modeling should be moving at least as fast as vision, audio, and robotics.

• Email: [email protected]
• X: @chemeris
• Telegram: @alexanderchemeris

The Core Intuition

A large temporal encoder can know more than its standard downstream head uses.

Suppose a classifier or forecaster appears strong on ordinary validation data but fails after a device replacement, site transfer, sampling-rate change, new workload, or regime transition. The failure may have two different causes:

1. the encoder never preserved the stable process state;
2. the stable state is present but awkward to select, while a shortcut is easier.

This idea targets the second case first. Instead of immediately retraining the foundation model, build a richer readout interface over its frozen representation:

$$\text{expanded state}=[\text{TSFM state};\text{temporal concepts};\text{concept residual}].$$

The purpose is not to certify causality. It is to give the downstream model explicit alternatives to shortcuts and then test those alternatives under shifts that break the shortcuts.

Stone-Level Picture

Imagine the frozen TSFM hands the readout one tangled box of signals:

real degradation + device identity + missingness + seasonality + site signature

A standard head chooses whichever combination best fits the training split. It may choose the site signature because that is easy.

The proposed interface adds labeled drawers:

trend | regime | missingness | device context | actions | residual

Now the head can more easily choose degradation and regime cues. The residual drawer keeps information that our named concept list missed.

The test is simple: when the site, device, time period, or regime changes, does the new head keep working while the old head fails?

Why Keep Both Concepts And Residuals

A pure concept bottleneck is dangerous for temporal data. A hand-built vocabulary will miss weak cross-channel relations, phase structure, delayed effects, and unknown failure precursors.

A pure dense embedding is also hard to audit. It can silently rely on tenant, device, cadence, preprocessing, or post-event leakage.

The combined interface aims for:

Block	Role	Main risk
Frozen TSFM state	Preserve the complete original representation	Still contains shortcuts
Temporal concepts	Expose named factors and typed context	Vocabulary may be incomplete or noisy
Residual	Preserve structure not predicted by concepts	Can retain or amplify the same shortcuts

Residual is therefore a completeness guard, not a robustness certificate.

Proposed Temporal Concept Schema

The first experiment SHOULD use deterministic or frozen concept extractors. It MUST NOT depend on another trainable large model whose adaptation would confound the readout result.

Channel	Examples
Level and shape	local level, slope, curvature, periodicity, phase, amplitude, volatility
Process state	operating regime, degradation, recovery, saturation, threshold proximity
Transition state	change point, transient, precursor, intervention response, delayed effect
Cross-channel structure	lag, synchrony, coherence, topology-local deviation, conservation residual
Data quality	missingness, censoring, clipping, quantization, resampling, stale sensor, unit/scale
Context	site, tenant, device, sensor family, workload, calendar, collection policy
Typed inputs	event, known exogenous variable, action, control input, intervention, treatment
Sensitive or prohibited proxies	demographic, commercial, or policy-sensitive attributes when applicable

The typed-input channel MUST preserve terminology:

• an event is not automatically an action;
• an action is a choice by an agent;
• a control input is an action entering system dynamics;
• an intervention is an intentional state-changing input in a causal or experimental frame;
• a treatment is a domain-specific intervention;
• an exogenous variable affects dynamics but is not chosen by the modeled agent.

Proposed Model

Let a frozen encoder produce $f(x)\in {\mathbb{R}}^d$ for a temporal window or trajectory prefix. Let deterministic/frozen concept extractors produce $c(x)\in {\mathbb{R}}^K$.

Fit a regularized map on the training partition:

$$W^{\ast }=\arg \underset{W}{\min }\Vert F-CW{\Vert }_F^2+\lambda \Vert W{\Vert }_F^2.$$

Define:

$$r(x)=f(x)-c(x{)}^{\top }{W}^{\ast }.$$

Unlike vision CRX, the first temporal version SHOULD compare several residual transforms:

1. no residual;
2. full residual;
3. PCA residual;
4. whitened residual;
5. supervised reduced-rank residual learned only on training data;
6. random orthogonal projection with the same dimension.

This is necessary because PCA keeps high-variance structure, which may be dominated by common devices or regimes while discarding rare but important state.

The Most Important Control

For full-rank linear residuals,

$$r=f-c^{\top }W,$$

so $r$ is determined by $f$ and $c$. Concatenating $[f;c;r]$ does not add new information beyond $[f;c]$. It changes coordinates and regularization geometry.

Therefore a temporal result MUST compare CRX-style expansion against:

• [f];
• [f;c];
• [f;r];
• [f;c;r];
• whitened [f;c];
• random-projection expansion matched in width;
• duplicated-feature expansion matched in width;
• an MLP on [f;c] matched in parameter count;
• blockwise regularization on [f;c] without residuals.

If [f;c;r] wins only against a weak linear [f;c] baseline, the result supports a useful coordinate system, not residual discovery.

Minimal Experiment

Stage 1: Controlled spurious temporal correlations

Create a synthetic multivariate dynamical system with:

• a target state or future regime;
• one stable predictive mechanism;
• one explicit shortcut that is correlated in training and reversed or removed at test time;
• rare regimes and transition windows;
• missingness and sampling artifacts;
• known actions or control inputs where relevant.

Example shortcut interventions:

• sensor ID predicts failure in training but not test;
• sampling cadence predicts regime only in training;
• site-specific scale predicts label and is swapped at test;
• post-event features leak the target and are removed at test;
• calendar period is correlated with load regime and then decorrelated.

Because the ground truth is known, this stage can distinguish robust state, nuisance factors, and leakage.

Stage 2: Public frozen-TSFM benchmark

Choose one classification/state-estimation task and one forecasting task. Freeze a public TSFM or JEPA encoder and build explicit environment splits:

• held-out site;
• held-out device or sensor family;
• held-out time period;
• held-out operating regime;
• shifted missingness or cadence;
• shifted workload or exogenous context.

All transforms—ridge, PCA, whitening, feature selection, and concept calibration—MUST be fitted only on the training partition. Hyperparameters MUST be selected on a separate validation environment or nested split, not on the same examples used to train the final readout.

Stage 3: Action-conditioned simulator

Only after passive tasks work, move to an action-conditioned simulator such as the proposed Kubernetes/OpenTelemetry control gym. Concepts should separate observed state, events, exogenous variables, actions, control inputs, and outcomes. Evaluate whether the readout improves candidate-action ranking or rollout state prediction under environment shift.

Observational correlations MUST NOT be described as intervention effects without support, overlap, or simulator evidence.

Evaluation

Average accuracy or MSE is insufficient.

Metric family	Required measurements
Group and environment robustness	worst-environment score, worst-regime score, held-out-site/device/time performance
Utility retention	mean score, per-task score, normal-regime calibration
Rare state	rare-regime recall, change-point sensitivity, precursor accessibility
Shortcut dependence	counterfactual shortcut swap, nuisance intervention, feature masking, conditional permutation
Representation access	linear/MLP probe performance, layer choice, residual effective rank
Stability	seed intervals, concept-weight stability, residual-subspace stability across environments
Cost	second-encoder cost, concept extraction, caching, readout training, inference latency and memory

A useful primary summary is a Pareto plot of mean utility against worst-environment utility. A method that raises the worst slice only by destroying ordinary behavior is not a free robustness gain.

Residual Discovery Loop

The residual can support ontology growth, but only under a strict rule:

A recurring residual direction becomes a candidate concept only if it is stable across environments, reproducible across seeds, tied to an observable temporal pattern, and useful after shortcut interventions.

flowchart LR
  X["Multivariate window"] --> E["Frozen TSFM"]
  X --> C["Temporal concept extractors"]
  E --> F["Dense state f"]
  C --> R["Concept-predictable state"]
  F --> R
  R --> U["Residual directions"]
  U --> H["Robust readout"]
  U --> S["Cross-environment stability tests"]
  S --> N["Candidate new concept"]
  N --> C

High variance alone is not enough. A device fingerprint can be a stable residual direction and still be exactly the shortcut we want to avoid.

Main Risks

• Incomplete concepts: useful state may remain hidden or be mislabeled.
• Residual shortcut leakage: residuals can contain site, device, background, or policy signatures.
• PCA mismatch: high variance can prioritize common nuisance factors and erase rare regimes.
• Non-stationarity: the concept-to-representation map $W^{\ast }$ can drift over time.
• Second-model confounding: gains may come from adding a stronger external encoder rather than concept structure.
• Validation leakage: fitting transforms and readouts on validation data can make model selection optimistic.
• Head-capacity confounding: a wider or nonlinear head can explain gains without a meaningful residual mechanism.
• Causal overclaim: better worst-environment prediction does not identify causal variables.
• Static-window limitation: a robust classifier head does not establish a streaming state, calibrated future distribution, or world model.

Success Criteria

The idea is worth pursuing if the expanded readout:

1. improves worst-environment or worst-regime performance across seeds;
2. preserves acceptable mean utility;
3. beats coordinate-, width-, and parameter-matched controls;
4. survives explicit shortcut swaps or removals;
5. uses concept channels stably enough to audit;
6. shows residual directions that are stable across environments rather than only high variance;
7. adds less cost than encoder fine-tuning for comparable robustness.

Reject the mechanism claim if a whitened or random coordinate expansion performs equally well. In that case, the practical trick may still be useful, but the correct conclusion is readout conditioning, not concept-residual discovery.

Relation To Existing Wiki Threads

• CRX is the direct vision precedent.
• Guillotine Regularization supports the broader claim that downstream transfer depends on where and how the representation is read.
• JEPA Slow Features warns that a non-collapsed predictive representation can preserve the wrong factors.
• Aionoscope motivates dense probes for amplitude, phase, frequency, timing, and regime rather than only component presence.
• Intermediate-Layer Representations suggests adding layer selection as an axis: CRX-style expansion may work better on an intermediate state than on the final embedding.
• Time-Series Benchmark Hygiene supplies the split, leakage, adaptation-mode, and uncertainty requirements.

Relation To Foundation TSFM Agenda

This is an idea page, so the verdicts below describe the intended contribution if the proposed experiment works. No temporal implementation or public result exists yet; current evidence status is stated separately in the Evidence and Missing pieces columns.

Agenda slot	Verdict	Evidence	Missing pieces
Representation quality	partially closes if validated	Proposes; not yet implemented. CRX supplies a concrete frozen-encoder readout precedent in vision, and adjacent sources show that readout layer/interface matters.	Temporal experiments proving stable state was present but inaccessible.
Context interface	partially closes if validated	Proposes; not yet implemented. Typed concepts would expose data quality, source context, events, exogenous variables, actions, and control inputs.	A schema and extractor suite that works across datasets without leaking labels.
Long tail and rare regimes	partially closes if validated	Proposes; not yet implemented. Worst-environment training/evaluation would make weak slices first-class.	Public rare-regime tests, seed intervals, and normal-retention curves.
Benchmark level	warning	Protocol warning, not a result. The claim depends on environments that actually break shortcuts.	Predeclared site/device/time/regime shifts, nested fitting, and matched controls.
Action-conditioned world models	adjacent	Proposes; not yet implemented. The same interface could audit state versus action/context channels.	Action-conditioned transition prediction, counterfactual support checks, and closed-loop utility.
Serving efficiency	adjacent	Proposes; not yet measured. Frozen encoders and a small head could be cheaper than full fine-tuning.	End-to-end latency and memory including any second encoder and online concept extraction.

Related Pages

• CRX
• Foundation Time-Series Model Research Agenda
• Intermediate-Layer Representations
• Time-Series Benchmark Hygiene
• Causal Time Series
• Action-Conditioned Time-Series Datasets
• Dynamic Curriculum Learning For JEPA
• Probabilistic JEPA Predictor Using Flow Matching