Gated DeltaNet

Summary

Gated DeltaNet is the linear recurrent attention architecture introduced by Gated Delta Networks: Improving Mamba2 with Delta Rule. It combines a data-dependent decay gate with the DeltaNet delta update rule so a fixed-size associative-memory state can both clear stale information and perform targeted key-value updates.

Role In The Wiki

Use this page as the object card for Gated DeltaNet. The source page carries the detailed evidence, limitations, and agenda mapping.

Gated DeltaNet matters locally because it is the parent architecture for Gated DeltaNet-2, a linear mixer used by Oryx, and a baseline/component in Hybrid Associative Memories. It is architecture background for long-context and recurrent-state time-series ideas, not direct TSFM evidence.

Evidence

• Gated Delta Networks: Improving Mamba2 with Delta Rule

Official Artifacts

• Preprint: arXiv 2412.06464
• OpenReview: ICLR 2025
• Official NVIDIA page: NVIDIA LPR publication page
• Official code: NVlabs/GatedDeltaNet

Related Pages

• Gated DeltaNet-2
• Oryx
• Hybrid Associative Memories
• Mamba-2
• Efficient Recurrent Sequence Models
• Extra-Long Context For Time Series
• Streaming Latent-State Updates
• Time-Series Scaling And Efficiency
• Foundation Time-Series Model Research Agenda