AnyMo Bench
Summary
AnyMo Bench is a public, fine-grained wearable IMU activity-recognition benchmark derived from real Nymeria streams. It evaluates unseen-subject generalization and same-position cross-device transfer using synchronized head, left-wrist, and right-wrist inertial measurements.
Dataset contract
| Field | Value |
|---|---|
| Source | Nymeria real in-the-wild wearable IMU |
| Subjects | 196: 157 train, 39 test |
| Body positions | head, left wrist, right wrist |
| Sampling | 60 Hz |
| Numeric features | 18 acceleration/gyroscope channels |
| Window length | 60-300 timesteps in the exported release |
| Label spaces | Fine150 and Core50 |
| Protocols | unseen subject; unseen subject + cross device |
| Public artifact | Hugging Face Parquet release |
| License surface | CC BY-NC 4.0 frontmatter plus Nymeria non-commercial terms |
Configurations
AnyMo-Bench-150-US: Fine150, first IMU for train and test.AnyMo-Bench-150-USCD: Fine150, first IMU for train and second co-located IMU for test.AnyMo-Bench-50-US: Core50, first IMU for train and test.AnyMo-Bench-50-USCD: Core50, first IMU for train and second co-located IMU for test.
Each configuration has 123,874 train rows and 30,965 test rows in the pinned artifact metadata.
Why it matters
AnyMo Bench isolates a practically important setup shift: the body location is held fixed while the physical IMU unit changes. The reported results show a large drop from unseen-subject to cross-device classification, so device invariance should be measured separately from subject invariance.
The benchmark also provides a stronger long-tail test than typical small-label wearable HAR suites: Fine150 keeps 150 activity classes, while Core50 merges them into a coarser but still substantial vocabulary.
Row-count tension
The paper and data card state 154,695 windows. Their split counts, and the pinned dataset_info.json, instead contain 123,874 train plus 30,965 test rows, totaling 154,839 per configuration. The wiki treats 154,839 as the exported artifact count while preserving 154,695 as the paper/card claim.
Benchmark hygiene
- Do not aggregate all four configurations as independent samples; they reuse the same window inventory.
- Report label space, subject protocol, and device protocol with every score.
- Keep Acc@1, Acc@5, and macro-F1 separate; long-tail performance is not captured by top-k accuracy alone.
- Pin the Hugging Face revision and label vocabularies.
- Distinguish exported variable-length windows from model-specific padding, splitting, or interpolation.
- Keep automated label construction, human review, and downstream evaluation as separate provenance layers.
Caveats
- One upstream dataset and three body positions.
- Cross-device evaluation uses co-located devices and does not cover arbitrary new placement.
- Label construction includes language-model proposal and adjudication.
- Fine150 and Core50 remain long-tailed.
- No forecasting, action, intervention, reward, or counterfactual interface.