Action100M
Summary
Action100M is a hierarchical open-vocabulary video-action annotation dataset introduced by an overlapping Meta FAIR/HKUST/Sorbonne team after VLWM. The full paper reports 1,199,096 HowTo100M videos and 147,092,653 temporally localized segment annotations. The public artifact is a 120,000-video 10% preview.
Official Artifacts
- Official Hugging Face preview: https://huggingface.co/datasets/facebook/action100m-preview
- Official GitHub: https://github.com/facebookresearch/Action100M
- Paper: https://arxiv.org/abs/2601.10592
- CVPR 2026 Workshop proceedings: https://openaccess.thecvf.com/content/CVPR2026W/EgoVis/html/Chen_Action100M_A_Large-scale_Video_Action_Dataset_CVPRW_2026_paper.html
- Dataset metadata snapshot: action100m-2026
Object Contract
Each released row identifies one YouTube source video and stores metadata plus a hierarchical list of time-bounded nodes. Node fields include PerceptionLM and Llama frame/segment captions and nullable GPT-OSS outputs for brief/detailed summaries, brief/detailed actions, and actor. Nodes shorter than four seconds remain in the hierarchy but are not passed through the final GPT aggregation stage.
The public schema contains no video-binary field. Action100M is therefore an annotation/metadata layer over external HowTo100M/YouTube sources.
Relationship To VLWM
The Action100M paper says its pipeline extends and improves the procedure introduced by VLWM. The distinction must remain explicit:
- Action100M is HowTo100M-only; VLWM reports six video datasets plus NaturalReasoning.
- Action100M labels segment actions, actors, and summaries.
- VLWM’s training schema adds a goal, goal interpretation, and explicit textual post-action state changes.
- Action100M is publicly previewed; the exact VLWM corpus is not.
Role In The Wiki
Action100M is the main large-scale video data-engine example for multi-level procedural action annotation. It is useful for studying hierarchical action semantics and automatically generated supervision, but observed action labels are events, not typed control inputs. Without before/after state, outcome, reward, or counterfactual alternatives, Action100M is not itself an action-conditioned world-model dataset.
Access And License
The 10% preview is public and ungated under the FAIR Noncommercial Research License. Commercial use is prohibited. Upstream source-video availability and rights remain separate. The full 147M-segment corpus is not publicly exposed through the official Hugging Face repository as of 2026-07-15.
Relation To Foundation TSFM Agenda
Use the source-level mapping in Action100M. At entity level, it is an adjacent data-engine and temporal-hierarchy example rather than a numeric time-series or control dataset.