Estimating Mutual Information between Time Series and Temporal Event Sequences Across Diverse Analysis Tasks

Source

• Raw Markdown: paper_humi-2026.md
• Paper PDF: paper_humi-2026.pdf
• Preprint: arXiv 2606.01602v3
• Version of record: ACM Digital Library
• Official code: HaojiHu/HUMI, audited at commit 2cee5be

The arXiv record was first submitted on 2026-06-01 and revised to v3 on 2026-06-19. ACM published the 12-page version of record on 2026-08-08 in the main proceedings of KDD 2026, pages 1746—1757. No separate official project page, official blog post, or author/lab X announcement was verified during ingest.

Status And Credibility

This is a current, peer-reviewed main-track paper at KDD 2026, a tier-1 data-mining venue. The ACM record reports a 21% KDD 2026 paper acceptance rate (1,157 of 5,395 submissions), provides DOI 10.1145/3770855.3817693, and releases the article under CC BY 4.0. The author team spans the University of Minnesota, University of Pittsburgh, and Inha University, and the paper provides an official implementation.

The credibility boundary is empirical and mathematical rather than venue-related. Ground-truth estimator accuracy is tested only on one synthetic construction and variants of its sample length/precision. Real-data evidence is indirect through seasonality, covariate ranking, and downstream classification. The appendix explicitly declines consistency and finite-sample error-bound analysis, and the released implementation materially differs from the paper’s clustering description.

Core Claim

A numeric time series and a categorical event sequence should not be forced into one homogeneous representation merely to measure dependence. Binning a time series creates arbitrary categories; assigning numeric IDs to event types creates an artificial order. The paper instead estimates mutual information directly across the two data types while treating repeated numeric values as probability mass rather than as broken continuous-nearest-neighbor cases.

The estimator has two main components:

1. Continuous—discrete duality for numeric observations: values that occur once are assigned to a continuous component; every occurrence of a repeated value is assigned to a discrete component.
2. Optional latent event clustering: event types with similar conditional distributions over aligned numeric values are merged before mutual-information estimation.

The official repository calls the implementation HUMI. The paper itself does not define HUMI as an acronym and usually calls it the proposed estimator or measurement.

flowchart LR
  S["numeric time series S"] --> A["align numeric values with events"]
  E["categorical event sequence E"] --> A
  A --> P["partition S by value multiplicity"]
  P --> C["unique values: continuous component"]
  P --> D["repeated values: discrete component"]
  A --> G["optional event clustering by conditional response distribution"]
  C --> M["weighted mixed-type mutual information"]
  D --> M
  G --> M
  M --> N["normalized dependence score"]
  N --> T["lag scan, repetition, covariate or feature ranking"]

Method

Let $E$ be a categorical event variable and $S$ the aligned numeric observation. The paper models the empirical numeric distribution as

$$p(s)=af(s)+bg(s),a+b=1,$$

where $f$ is the continuous component and $g$ is the discrete probability mass over repeated values. It then defines the mixed estimator as

$$\hat{I}(E;S)=a\hat{I}(E_A;S_A)+b\hat{I}(E_B;S_B)+a\log a+b\log b,$$

where the $A$ subset contains values seen once and the $B$ subset contains all occurrences of values seen more than once. The continuous term uses a one-dimensional Kozachenko—Leonenko-style nearest-neighbor entropy estimate; the discrete term uses empirical frequencies. Negative estimates are clipped to zero.

The reported normalized score is

$$\mathrm{NMI}(E,S)=\sqrt{1-\exp \left(-\frac{2\hat{I}(E;S)}{\Delta E+\Delta S}\right)},$$

with the discrete-symbol unit approximated as $\Delta E=1$. The paper presents this as a bounded, correlation-like score, although the normalization is not a standard discrete—continuous NMI and needs stronger invariance and calibration analysis.

For latent event clustering, the paper says it:

1. collects the numeric values aligned with each event type;
2. compares the resulting empirical conditional distributions using Wasserstein distance;
3. applies hierarchical clustering;
4. replaces event types with cluster labels before recomputing MI.

This clustering is useful when several nominal event types induce nearly the same numeric response. It also makes the dependence estimate partly adaptive to the observed target series rather than a fixed event ontology.

Evaluated Tasks

Task	Numeric side	Event side	What is measured
Time-delayed MI	synthetic numeric series	three event types	lagged dependence at candidate delay $\tau$
Global seasonality	Minneapolis daily traffic volume	day-of-week labels	strength of weekly repetition
Local repetition	12-hour temperature observations	day/night and two-month labels	context-conditioned reduction in numeric uncertainty
Forecast covariate selection	Rossmann and M5 sales	promotions, holidays, calendar/product categories	agreement between MI ranking and downstream forecast ranking
Tabular feature selection	continuous features	class labels	downstream classification after selecting top-$k$ features

The tabular experiment is not a temporal-sequence result: instance order is irrelevant there. It demonstrates that the same mixed-type estimator can rank continuous features against categorical labels.

Main Evidence

Controlled time-delay recovery

The synthetic construction has 10,000 event observations, three equally likely event types, event-specific Gaussians, two-decimal numeric precision, and a true delay of $\tau =5$. At the true lag:

Estimator	Estimate at $\tau =5$	MSE to reported ground truth $1.0984$	Detects peak?
Ross	-3.1468	18.022	No usable nonnegative estimate
Discrete—continuous Mixture baseline	1.1078	$9\times 10^{-5}$	Yes
HUMI	1.0988	reported as 0 at five-decimal precision	Yes

Across the ten tested lags, HUMI is closer to the paper’s analytic/Taylor ground truth than the Mixture baseline. Appendix experiments vary decimal precision and sample count, but remain variants of the same three-Gaussian construction rather than independent distribution families.

Repetition and seasonality

For Minneapolis traffic, HUMI reports weekly-seasonality scores of 0.6501 in June—July and 0.5664 in December—January, matching the paper’s qualitative claim that holiday disruption weakens the pattern. For the local temperature example, the score rises monotonically as the event context becomes more informative:

$$0.6124\text{(day/night)}\to 0.8932\text{(two-month block)}\to 0.9596\text{(joint context)}.$$

This is evidence that the score tracks increasing calendar-conditioned separability in this dataset. It is not a calibrated universal rule that NMI > 0.5 means seasonality; that threshold is used heuristically in the experiment.

Discrete covariate ranking for forecasting

The paper samples 100 series from each of Rossmann and M5 and compares MI-based covariate rankings with downstream ranking under CatBoost, DeepAR, FMTimeSeries/TimesFM, and Chronos-2. The clustered estimator has the highest mean NDCG in all eight reported model—dataset cells:

Forecast model	Rossmann NDCG	M5 NDCG
CatBoost	0.95	0.86
DeepAR	0.92	0.82
FMTimeSeries / TimesFM	0.96	0.92
Chronos-2	0.95	0.83

These results support HUMI as a pre-model context/covariate filter. They do not show that MI-selected covariates always improve forecasting relative to using every covariate, nor that dependence implies causal usefulness.

Continuous feature selection

Across 120 dataset—classifier—feature-budget cells, the unclustered estimator records 89 wins, 14 ties, and 17 losses against Ross; and 93 wins, 15 ties, and 12 losses against the Mixture baseline. Adding clustering records 77 wins, 20 ties, and 23 losses against the unclustered variant. The aggregate is positive but not uniform; several individual cells favor a baseline.

Independent Artifact Audit

The official repository is useful but not yet a clean reproduction package.

• The paper describes Wasserstein distance for event-distribution clustering. The pinned normalize_clustered_mi.py imports Wasserstein distance but actually builds the pairwise matrix with SciPy energy_distance.
• Appendix E says the hierarchical-clustering distance threshold is fixed to 0.9. The code instead computes the 90th percentile of pairwise distances when percentile=90; these are different hyperparameter semantics. The accompanying sensitivity script also labels its x-axis as percentile.
• Equation 12 and the paper pseudocode include the mixture-weight terms $a\log a+b\log b$ in the reported MI formula. The pinned mixture_mi.py does not implement that expression: it computes $H(E)+a(H(S_A)-H(E_A,S_A))+b(H(S_B)-H(E_B,S_B))$ and contains no mixture-log terms. This may reflect a correction in code, but the repository does not document the reconciliation, so paper-equation and code results MUST be treated as different estimator specifications until clarified.
• The paper’s pseudocode applies clustering only “if correlation among events is observed.” NormalizedClusteredMI always clusters; selecting clustered versus unclustered behavior is delegated to which function the experiment calls.
• The repository has no detected license file, no tagged release, no package metadata, and no focused test suite. Its requirements.txt is a 225-package environment snapshot that includes multiple CUDA runtime families rather than a minimal reproducible dependency set.
• The code rescales numeric values by their observed range without an explicit constant-series guard. Constant or degenerate inputs therefore need separate handling in production use.
• The GitHub repository URL redirects from the long paper-derived name to HaojiHu/HUMI; the canonical repository name should be used for pinned citations.

Limitations And Gotchas

• Dependence is not causality. Time-delayed MI can expose lagged statistical dependence, but a peak does not identify a causal direction under common causes, autocorrelation, selection effects, or calendar confounding.
• Multiplicity is sample-dependent. The unique-versus-repeated split changes with sample length, numeric precision, units, rounding, and noise. A value observed once is not evidence that its generating component is truly continuous.
• Clustering is target-dependent. Event types are merged from their conditional numeric-response distributions. This can improve estimation, but the resulting event ontology may change across targets, windows, tenants, or regimes.
• No uncertainty estimates are reported. Point estimates, mean NDCG, and win counts are reported without confidence intervals or a full multi-seed estimator-variance study.
• Negative clipping can hide estimator failure. Enforcing $\max (\hat{I},0)$ restores the theoretical range but does not diagnose finite-sample bias or calibration near independence.
• The normalization is heuristic. The paper’s $[0,1]$ transform approximates the discrete-symbol unit and is not shown to be invariant or calibrated across event cardinalities, precision levels, and sample sizes.
• The appendix proof is not sufficient as written. Nonnegativity of true mixed-variable MI follows from KL divergence, but the proof invokes a joint-entropy inequality that is not generally valid as stated; it does not establish nonnegativity, consistency, or finite-sample bounds for the proposed estimator.
• The printed mixture formula needs reconciliation. The $a\log a+b\log b$ terms are nonpositive and do not appear in the official implementation. The paper does not explain whether the equation, the pseudocode, or the code defines the reported experiment values.
• Ground truth is narrow. Only the synthetic TDMI setup has stated ground-truth MI. Seasonality and local repetition use visual/physical intuition; covariate and feature selection use downstream task performance as a proxy.
• Model-selection comparison is incomplete. The forecasting experiment evaluates ranking quality, not the end-to-end delta versus all-covariate, no-covariate, conditional-MI, redundancy-aware mRMR, or learned-gating baselines under matched tuning budgets.

Foundation TSFM Relevance

Agenda slot	Verdict	Evidence	Missing pieces
Context interface	partially closes	Provides a model-agnostic operator for ranking categorical events and covariates against numeric targets before a TSFM consumes them.	Needs conditional/redundancy-aware selection, multivariate target state, out-of-sample selection stability, and end-to-end all-covariate baselines.
Time representation and event streams	partially closes	Directly compares categorical event sequences with numeric observations and scans event-to-observation lags without assigning ordinal IDs to events.	Alignment is preconstructed; no high-rate asynchronous event parser, duration model, irregular multivariate state, or streaming update is evaluated.
Benchmarks	warning	Exposes quantization, repeated values, event redundancy, and transformation bias as first-order evaluation variables.	Needs broader synthetic ground truths, conditional-independence cases, confidence intervals, seeds, calibration, and released test harnesses.
Causal structure	warning	Time-delayed dependence can screen candidate lead—lag relationships.	No confounder adjustment, conditional MI, intervention, direction-identification theorem, or counterfactual evaluation.
Control and counterfactuals	insufficient evidence	Calendar and promotion variables are analyzed as observed events/covariates.	No action, control input, intervention, or candidate-action rollout is present.

Links Into The Wiki

• HUMI
• Heterogeneous Temporal Dependence
• Context-Aided Forecasting
• Causal Time Series
• Time-Series Foundation Models
• Time-Series Benchmark Hygiene
• Foundation Time-Series Model Research Agenda
• Contradictions And Open Tensions

Open Questions

• Does the estimator remain calibrated when numeric precision changes over time or across channels rather than being globally fixed?
• Should repeated observations be modeled with an explicit measurement-resolution/noise model instead of a hard sample-multiplicity split?
• Can conditional MI distinguish one event’s incremental information after seasonality, other events, and correlated covariates are accounted for?
• Does target-dependent event clustering transfer across series, tenants, regions, and regimes, or does it leak target-specific structure into the event ontology?
• How does HUMI compare with all-covariate training, learned sparse gates, mRMR-style redundancy penalties, conditional-independence tests, and permutation importance under matched tuning cost?
• Can an online estimator update dependence and clustering under drift without recomputing over the full historical stream?