Expectations vs. Realities: The Cost of MSE-Optimal Forecasting Under Conditional Uncertainty

Source

• Raw Markdown: paper_mse-optimal-forecasting-conditional-uncertainty-2026.md
• Paper PDF: paper_mse-optimal-forecasting-conditional-uncertainty-2026.pdf
• Preprint: arXiv 2606.04342v1
• Version of record: ACM DOI 10.1145/3770855.3818087

Status And Credibility

Riku Green, Zahraa S. Abdallah, and Telmo M Silva Filho submitted arXiv v1 on 2026-06-03. The three authors are affiliated with the University of Bristol. The archival version was published on 2026-08-08 in volume 2 of the Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1298–1309; KDD 2026 ran on 2026-08-09–13 in Jeju. This is a peer-reviewed KDD Research Track paper at a tier-1 data-mining and machine-learning venue. ACM reports a KDD 2026 paper acceptance rate of 1,157 / 5,395 submissions (21%).

The user supplied a photograph of the KDD 2026 poster, so this source defaults to importance: important and read_status: skimmed. No clearly official code repository, project page, blog post, or author/lab X announcement was verified as of 2026-08-11. The arXiv source archive is complete enough to inspect the derivations, experimental protocol, and camera-ready figures.

Core Claim

For long-horizon forecasting under irreducible conditional uncertainty, the mean-squared-error-optimal deterministic prediction is the conditional mean. That prediction is optimal for squared error but systematically loses the part of future variability that remains unresolved by the observed history. Therefore it can be smooth and accurate on average while being unrepresentative of typical realized futures.

The paper formalizes this as an intrinsic accuracy–realism frontier, not as an architecture defect:

flowchart LR
  H["observed history X"] --> D["conditional future distribution"]
  D --> M["conditional mean: lowest MSE"]
  D --> S["sample / variability-preserving rule"]
  M --> U["under-dispersed marginal predictions"]
  S --> R["more realistic marginal variability"]
  U --> P["accuracy–realism Pareto frontier"]
  R --> P

A probabilistic model does not make the decision disappear. Reporting its predictive mean, drawing one sample, or using another inference rule chooses a different operating point on the same frontier.

Conditional Uncertainty Gap

For history $X$ and the scalar target $Y_h$ at horizon $h$, the law of total variance gives

$$\mathrm{Var}(Y_h)=\mathrm{Var}(\mathbb{E}[Y_h\mid X])+\mathbb{E}[\mathrm{Var}(Y_h\mid X)].$$

The paper defines the horizon-dependent conditional uncertainty gap

$${\mathrm{Gap}}_h=\frac{\mathbb{E}[\mathrm{Var}(Y_h\mid X)]}{\mathrm{Var}(Y_h)}=1-\frac{\mathrm{Var}(\mathbb{E}[Y_h\mid X])}{\mathrm{Var}(Y_h)}.$$

Gap is the fraction of marginal target variance that cannot be explained by the observed history. When it is positive, the Bayes predictor

$$f_h^{\ast }(X)=\mathbb{E}[Y_h\mid X]$$

has strictly lower marginal variance than $Y_h$. It therefore cannot have the same marginal law as the realized target. Any deterministic predictor that does match the target marginal must differ from $f_h^{\ast }$ on a set of positive probability and incur strictly larger MSE.

This result is deliberately modest in scope. It proves an incompatibility between MSE optimality and marginal distribution matching. It does not say that one sample from a probabilistic forecaster is conditionally correct, calibrated, temporally coherent, or decision-usable.

What Marginal Realism Measures

Standard deterministic forecasting datasets usually provide one realized future for each context. They do not expose repeated samples from $p(Y_h\mid X=x)$ for the same real context. The paper therefore compares, for each horizon, the empirical marginal distributions across contexts:

$${\hat{P}}_h^f=\frac{1}{N}\sum _{i=1}^N{\delta }_{f_h(X_i)},{\hat{P}}_h^Y=\frac{1}{N}\sum _{i=1}^N{\delta }_{Y_{h,i}},$$

and measures their distance with horizon-wise one-dimensional Wasserstein distance $W_1$.

This is useful because it exposes conditional-mean under-dispersion with only one prediction and one realization per context. It is also the weakest nontrivial realism test in this setting:

• matching marginal $W_1$ does not establish conditional calibration;
• it does not establish that samples correspond to the correct context;
• it ignores temporal order and cannot establish trajectory realism;
• it does not determine coverage, sharpness, tail risk, or mode probabilities.

The paper therefore treats Dynamic Time Warping, vector Wasserstein, order-preserving Wasserstein, fused Gromov–Wasserstein, interval coverage, and interval width as complementary diagnostics rather than substitutes for a full probabilistic evaluation.

Experimental Design

Controlled System

The controlled experiment uses a stochastic Mackey–Glass system with adjustable additive state noise. For each selected context, 1,000 independent stochastic rollouts approximate the conditional distribution. This permits direct estimation of Gap and shows that the attained Pareto front becomes longer as the horizon and conditional uncertainty increase.

Real Benchmarks

The main real-data protocol uses nine datasets: ETTh1, Electricity, METR-LA, PEMS04, PEMS08, PEMS-BAY, Weather, BeijingAirQuality, and ExchangeRate. Important scope details are:

• only the first seven variates are used, each as a separate univariate forecasting task;
• lag length is 20 and the maximum forecast horizon is 200;
• each series uses an 80/20 chronological split and training-set standardization;
• 1,000 test windows are sampled per task;
• the resulting analysis contains $9\times 7\times 200=12,600$ horizon-specific Pareto fronts.

The main candidate set combines recursive and direct multi-output strategies with linear ridge regression, MLP, decision-tree, and conditional normalizing-flow predictors. The paper explicitly prioritizes coverage of operating points over a new state-of-the-art leaderboard. A stronger-pool appendix adds LSTM, N-BEATS, PatchTST, TCN, and probabilistic variants on a smaller subset of datasets. A separate robustness study uses zero-shot Chronos-T5 tiny, mini, and small models, horizons up to 50, five sampling temperatures, and 100 samples per context.

Main Results

Five-Percent MSE Tolerance Band

The practical selection rule retains candidates satisfying

$$\mathrm{MSE}(f)\le (1+ϵ){\mathrm{MSE}}_{\min },$$

then chooses among those near-ties using marginal $W_1$, with calibration and sharpness checks for probabilistic forecasts. At $ϵ=5\%$ across the nine real benchmarks, the paper reports:

• median attainable marginal-realism improvement: 17.3%;
• median probability that the realism gain exceeds the MSE loss: 30.6%;
• at least some materially better near-MSE-optimal alternatives in every dataset.

The effect is heterogeneous. Weather reports a 22.6% median realism improvement with a 38.4% success probability; ExchangeRate reports 23.3% with 24.4%; PEMS08 reports a larger 27.7% median gain but only a 13.2% success probability.

Strategy And Inference Positioning

Across the 12,600 main Pareto fronts:

• direct multi-output point predictors occupy 60% of the MSE-optimal extremes on average;
• recursive point predictors occupy 39% of the marginal-$W_1$-optimal extremes;
• direct multi-output probabilistic models with single-sample inference occupy 49% of the marginal-$W_1$-optimal extremes;
• direct probabilistic single-sample inference never occupies the MSE-optimal extreme in the reported sweep.

These are empirical tendencies for the tested pool, not universal rankings. Their value is to show that strategy and inference rules act as practical controls over which functional of the future distribution is emphasized.

Practical Reading

The paper does not argue that MSE should be discarded. It argues for constrained multi-objective model selection:

1. find the best validation MSE;
2. define an application-dependent tolerance $ϵ$;
3. retain near-tied candidates inside that MSE band;
4. compare marginal realism within the band;
5. for probabilistic forecasts, also check proper scores, conditional calibration, coverage, sharpness, tails, and trajectory coherence.

The tolerance should reflect the downstream cost. For simulation, scenario generation, stress testing, and long-horizon decision support, a small point-error increase may be acceptable when it restores missing variability. For decisions where small point-error differences are operationally decisive, $ϵ$ should be close to zero.

Relation To Probabilistic Future Modeling

This source gives direct numeric time-series evidence for why a single average future is insufficient. It strengthens the motivation for Probabilistic JEPA Predictor Using Flow Matching, but it does not establish that flow matching, JEPA latents, or any other specific architecture is the right solution.

The paper’s tolerance-band protocol is best used as an early evaluation layer:

• first test whether a method restores horizon-wise variability without an unacceptable MSE cost;
• then test whether it preserves separated regimes, correct per-context samples, calibrated mode mass, rare tails, multivariate constraints, and action-sensitive probability shifts.

A model can pass the marginal test while permuting futures among contexts or generating temporally invalid trajectories. Marginal realism is necessary evidence against over-smoothing, not sufficient evidence for a calibrated predictive belief.

Foundation TSFM Relevance

Agenda slot	Verdict	Evidence	Missing pieces
Multi-modal future distributions	warning	Proves that a deterministic conditional mean is under-dispersed whenever conditional uncertainty is nonzero and shows the frontier directly on numeric time-series benchmarks.	Does not identify or calibrate separated future modes, rare branches, or invalid between-mode trajectories.
Benchmarks: what level of modeling is tested	partially closes	Supplies a concrete MSE-tolerance-band protocol, marginal $W_1$ diagnostic, Pareto reporting, and calibration/sharpness boundary.	Needs standardized conditional, trajectory, tail-risk, multivariate, and decision-utility evaluation across stronger current forecasters.
Latent-state prediction	warning	Demonstrates that low observation-space MSE can coexist with an unrepresentative future summary.	No latent-state model or probe is evaluated.
Native multivariate encoding	insufficient evidence	Uses real multivariate datasets as source material.	Treats the first seven variates as separate univariate tasks and does not test cross-channel constraints.
Control and counterfactuals	insufficient evidence	Notes simulation, planning, and decision support as use cases.	No action, control-input, intervention, counterfactual, or planner-utility experiment.

Limitations And Gotchas

• The core theorem concerns marginal law matching for deterministic predictors; it is not a complete theory of stochastic predictive distributions.
• Marginal $W_1$ can be good even if predictions are assigned to the wrong contexts or trajectories have invalid temporal order.
• The main real-data experiment is channel-independent/univariate, so it does not test native multivariate constraints or high-channel dynamics.
• The main candidate pool is designed to span operating points rather than establish a state-of-the-art leaderboard.
• The stronger-candidate appendix covers only a subset of datasets, and the Chronos robustness study uses three Chronos-T5 sizes with a shorter horizon protocol.
• Calibration and interval-width diagnostics are complementary and heterogeneous; improved marginal realism must not be rewritten as universal calibration improvement.
• Gap can be estimated cleanly in the controlled simulator through repeated rollouts, but not from ordinary real-world single-future datasets. The practical tolerance-band method does not require estimating it.
• No official implementation or full reproduction package was verified.
• The source is passive forecasting evidence, not an action-conditioned world model.

Links Into The Wiki

• Probabilistic JEPA Predictor Using Flow Matching
• Latent-State Time-Series Modeling
• Time-Series Benchmark Hygiene
• Time-Series Generation
• Foundation Time-Series Model Research Agenda
• Contradictions And Open Tensions

Open Questions

• Which conditional or trajectory metric is reliable when only one realized future is available for each context?
• How should the MSE tolerance be selected from downstream decision cost rather than fixed globally at 5%?
• Do current foundation forecasters exhibit the same frontier under matched sampling, calibration, and compute budgets?
• Can a model improve marginal realism without assigning plausible trajectories to the wrong contexts?
• How should separated-mode coverage, probability mass, tail risk, and multivariate constraints be layered on top of the paper’s inexpensive marginal diagnostic?
• Does better navigation of the accuracy–realism frontier improve action ranking, stress testing, or closed-loop control?