Energy-Based Transformer

Summary

Energy-Based Transformer (EBT) is a Transformer implemented as an explicit energy-based model. It scores the compatibility between a context and a candidate prediction, then generates by minimizing that energy with respect to the candidate prediction.

Role In The Wiki

EBT is the current source-backed anchor for energy-based dynamic compute in modern Transformer-like architectures. Its strongest wiki value is the interface pattern:

context + candidate prediction
  -> energy / compatibility score
  -> gradient-based candidate refinement
  -> optional self-verification over candidate predictions

For the time-series agenda, EBT is not yet direct evidence: the paper does not evaluate numeric time series or action-conditioned operational systems. It is still important because it offers a concrete mechanism for spending extra compute on hard predictions, scoring candidate futures, and potentially optimizing candidate actions in an energy-scored world model.

Official Artifacts

• Official project website: https://energy-based-transformers.github.io/
• Official blog post: https://alexiglad.github.io/blog/2025/ebt/
• Official code: https://github.com/alexiglad/EBT
• Preprint: https://arxiv.org/abs/2507.02092

Evidence

• Energy-Based Transformers are Scalable Learners and Thinkers

Relation To Foundation TSFM Agenda

Use the source-level agenda mapping in ebt-2025 rather than duplicating verdict rows here.

At the entity level, EBT is the object card for the method family. The source page carries the slot-level verdicts, evidence, limitations, and missing pieces.

Related Pages

• Energy-Based Models
• Slow Thinking For Robotics And Time Series
• Foundation Time-Series Model Research Agenda
• Time-Series Scaling And Efficiency
• Unified Multimodal Models
• World Models