Alex Open Research Wiki

AnyMo

3 min read

AnyMo

Summary

AnyMo is a geometry-aware model for understanding human motion from sparse wearable inertial measurement units under changes in body placement, mounting orientation, device hardware, and dataset. Its core design treats setup variation as structured body-surface geometry rather than arbitrary domain noise.

It combines physics-grounded synthetic IMU generation, a 23-node body-graph encoder trained to infer full-body motion from one to five visible sensors, product-quantized IMU tokens, and a Qwen2.5-0.5B motion-language model for zero-shot activity recognition, IMU-text retrieval, and motion captioning.

Pipeline

flowchart LR
  Motion[Nymeria body motion + mesh] --> Sim[dense geometry-aware IMU simulation]
  Sim --> Views[paired setup views on a 23-node body graph]
  Views --> Encoder[masked cross-view ST-GCN encoder]
  Encoder --> Tokens[PQ-VAE full-body IMU tokens]
  Tokens --> LLM[Qwen2.5 motion-language alignment]
  LLM --> HAR[zero-shot activity recognition]
  LLM --> Retrieval[IMU-text retrieval]
  LLM --> Caption[motion captioning]

Model contract

  • Input: one or more sparse wearable IMU streams plus mapped body-location identifiers.
  • Numeric features: three-axis acceleration and angular velocity.
  • Graph context: a fixed 23-node body topology based on the Nymeria/Xsens kinematic tree.
  • Representation: a time-preserving ST-GCN latent, discretized by two 2,048-entry product codebooks.
  • Language backbone: Qwen2.5-0.5B with projected IMU code vectors and contrastive/instruction tuning.
  • Outputs: pooled motion-language embeddings or generated text.

Reported evidence

The paper reports average zero-shot results across 14 unseen activity-recognition datasets of 35.7 accuracy, 29.5 macro-F1, and 57.5 R@2, marked as 11.7%, 11.6%, and 22.6% improvements over the comparison reference used by the authors. It also reports relative MRR gains of 15.9% for IMU-to-text retrieval and 28.6% for text-to-IMU retrieval, plus an 18.8% relative gain in captioning BERT-F1.

These are author-reported preprint results. The public repository was README-only at commit 0f0d989f59f517658feb8c651a2bbc48e88eaa9c, with no code, weights, license file, tags, or releases as of 2026-08-10.

Role in the wiki

AnyMo is a concrete wearable-sensing example of three broader time-series ideas:

  1. use known geometry and physics to synthesize nuisance variation rather than applying arbitrary augmentations;
  2. train representations to recover a structured full state from sparse multivariate observations;
  3. expose compact numeric tokens to a language model for open-vocabulary retrieval and generation.

It is not an action-conditioned world model. The paper evaluates passive motion understanding rather than forecasting, interventions, control inputs, rewards, or closed-loop decisions.

Artifact boundary

  • Paper and project page: public.
  • AnyMo Bench: public on Hugging Face under non-commercial terms.
  • Code repository: public placeholder only at the pinned audit revision.
  • Model weights and executable training/inference pipeline: not publicly released at the audit date.

Caveats

  • Recent arXiv preprint with no peer-reviewed venue identified.
  • Real motion-text supervision and evaluation are centered on Nymeria and wearable HAR.
  • Synthetic placement coverage depends on the body-mesh/skeleton and device-noise assumptions.
  • Captioning and retrieval scores do not establish dense motion-state reconstruction or calibrated uncertainty.
  • The AnyMo Bench headline window count conflicts with its published split and artifact metadata.

Graph View

Backlinks