Titans: Learning to Memorize at Test Time

Source

• Raw Markdown: paper_titans-2025.md
• PDF: paper_titans-2025.pdf
• Preprint: arXiv 2501.00663

Core Claim

Titans introduces a neural long-term memory module trained to memorize historical context at test time and combines it with attention as short-term memory.

Relevance To This Wiki

This is the main recent memory-side successor in the Universal Transformer neighborhood: it treats context as a stateful resource updated during inference, rather than only as a fixed attention window.

Limitations

The strongest claims are broad sequence-model claims; direct multivariate time-series evidence should be separated from language, genomics, and synthetic long-context results.

Foundation TSFM Relevance

Relevant to streaming state and long-context memory for time-series models, especially if memory updates can preserve numeric regimes and exogenous context over long horizons.

Links Into The Wiki

• Titans
• Looped Transformers And Test-Time Memory
• Efficient Recurrent Sequence Models
• Time-Series Scaling And Efficiency
• Titans Revisited
• Foundation Time-Series Model Research Agenda

Open Questions

• What matched-budget baseline should this source be compared against: unique-depth Transformer layers, recurrent state, explicit memory, or extra inference steps?
• Which claims transfer from token-sequence reasoning to multivariate time-series state tracking, event streams, or action-conditioned world models?