Alex Open Research Wiki

Implicit Maximum Likelihood Estimation (IMLE)

3 min read

Implicit Maximum Likelihood Estimation (IMLE)

Summary

Implicit Maximum Likelihood Estimation is a family of likelihood-free objectives for implicit generators. It draws model samples, assigns every observed datapoint or conditional target to a nearby generated sample, and updates the generator through those data-to-sample matches. This assignment direction discourages mode dropping without an adversarial discriminator.

Canonical Forms

Original unconditional IMLE

A global pool of generated samples is shared by the dataset:

The primary source is the 2018 rejected NIPS/arXiv preprint Implicit Maximum Likelihood Estimation.

Conditional IMLE

Each conditioning input receives its own latent candidates:

The peer-reviewed primary source is the IJCV 2020 article Multimodal Image Synthesis with Conditional IMLE.

Relation To Explorative Models

The hard loss of direct conditional Forward XM is the conditional-IMLE objective under different notation. The nearest-of- estimator therefore predates XM.

Explorative Modeling contributes a broader interpretation and experimental program rather than the basic winner-take-all assignment: candidate width as training-time generative expressivity, applications to intermediate predictions inside existing factored generators, a Forward/Reverse taxonomy, and scaling experiments across modalities.

The two inclusion claims should be scoped:

  • “IMLE is an end-to-end Forward XM instance” is true under XM’s umbrella taxonomy, especially for original IMLE’s shared global pool.
  • “End-to-end conditional Forward XM is cIMLE” is true at the hard-objective level.
  • “All XM is just IMLE” is too strong because hybrid and Reverse XM alter the candidate construction or matching direction.
  • “XM invented best-of-” is false; the XM paper itself disclaims that claim.

Evidence Boundary

Original IMLE’s exact MLE-equivalence theorem is conditional; the practical unweighted objective needs an equal-optimal-density condition for the paper’s exact identity. Conditional IMLE supplies strong historical evidence for multimodal conditional image generation, but not modern time-series, calibrated probability, or action-conditioned planning evidence.

Hard IMLE-family losses primarily certify proposal coverage. Candidate frequencies MUST NOT be treated as calibrated event probabilities without a separate proper scoring objective or calibration evaluation.

Role In The Wiki

IMLE is the historical root for direct candidate-width generation. For multivariate time series, it provides a baseline against deterministic mean futures:

  • condition on history and context;
  • sample multiple future trajectories;
  • train through the closest trajectory;
  • separately evaluate coverage, validity, calibrated mode mass, tail risk, and action-conditioned planning utility.

Official Artifacts

Graph View

Backlinks