Latent Context Language Models

Summary

Latent Context Language Models are encoder-decoder soft-token compressors for long-context language models. They compress raw prompt spans into latent tokens before decoder prefill, preserving the standard decoder interface while reducing context length, memory, and time to first token.

Interface

• Input: long prompt or context token sequence.
• Compression unit: fixed encoder windows whose token blocks are pooled into latent tokens.
• Decoder contract: generated answers condition on projected latent tokens instead of the full raw prompt.
• Agent extension: an EXPAND(i) tool can recover raw text for selected compressed chunks.
• Current artifact status: arXiv preprint, ICML 2026 SPIGM workshop OpenReview page, official code, and Hugging Face checkpoints.

Role In The Wiki

Use this page as the object card for LCLM. The source page carries the evidence, caveats, and time-series relevance mapping.

For the foundation time-series agenda, LCLM is upstream evidence for learned context compression and compressed agent memory. It should not be cited as direct evidence that numeric time-series latent state, event-stream state, or action-conditioned history survives compression.

Evidence

• End-to-End Context Compression at Scale

Official Artifacts

• Preprint: arXiv 2606.09659
• OpenReview: ICML 2026 SPIGM workshop poster
• Code: LeonLixyz/LCLM
• Hugging Face: latent-context
• X thread: Micah Goldblum post

Related Pages

• Time-Series Scaling And Efficiency
• Latent Tokenization
• Looped Transformers And Test-Time Memory
• Foundation Time-Series Model Research Agenda
• Compress & Attend Transformer
• TurboQuant