--- abstract: | As wearable and mobile devices become increasingly embedded in daily life, they offer a practical way to continuously sense human motion in the wild. But inertial signals are highly dependent on the sensing setup, including body location, mounting position, sensor orientation, device hardware, and sampling protocol. This setup dependence makes it difficult to learn motion representations that transfer across devices and datasets, and limits the broader use of wearable IMUs beyond closed-set recognition. We introduce AnyMo, a geometry-aware framework for setup-agnostic human motion modeling. AnyMo uses physics-grounded IMU simulation over dense body-surface placements to generate diverse and plausible synthetic signals, pre-trains a graph encoder from paired synthetic placement views and masked partial observations, tokenizes multi-position IMU into full-body motion tokens, and aligns these tokens with an LLM for motion-language understanding. We evaluate AnyMo on three complementary tasks: zero-shot activity recognition across 14 unseen downstream datasets, cross-modal retrieval, and wearable IMU motion captioning, where it improves average Accuracy/F1/R@2 by 11.7%/11.6%/22.6% on HAR, increases zero-shot IMU-to-text and text-to-IMU retrieval MRR by 15.9% and 28.6%, respectively, and improves zero-shot captioning BERT-F1 by 18.8%. These results support AnyMo as a generalist model for wearable motion understanding in the wild. Project page: . author: - | Baiyu Chen$^{1}$[^1] `\quad`{=latex} Zechen Li$^{1}$ `\quad`{=latex} Wilson Wongso$^{1}$ `\quad`{=latex} Lihuan Li$^{1}$ `\quad`{=latex} Xiachong Lin$^{1}$\ **Hao Xue$^{1,2,3}$ `\quad`{=latex} Benjamin Tag$^{1}$ `\quad`{=latex} Flora Salim$^{1\ast}$**\ $^1$The University of New South Wales\ $^2$The Hong Kong University of Science and Technology (Guangzhou)\ $^3$The Hong Kong University of Science and Technology bibliography: - ref.bib title: | AnyMo: Geometry-Aware Setup-Agnostic\ Modeling of Human Motion in the Wild --- ```{=latex} \def\sectionautorefname{Section} ``` ```{=latex} \def\subsectionautorefname{Section} ``` ```{=latex} \def\subsubsectionautorefname{Section} ``` ```{=latex} \maketitle ``` # Introduction {#sec:introduction} Human motion is one of the most immediate expressions of human context in everyday life [@rosenhahn2008human; @aggarwal2011human]. When people walk, cook, exercise, or interact with others or objects, their movement directly reflects their engagement with their surroundings over time. Understanding this context is important for future proactive AI and human-centered computing systems, which must proactively respond to changing user contexts and adapt in real environments rather than simply waiting for explicit commands [@schilit1994context; @dey2001understanding]. The growing ubiquity of wearable and mobile devices, from watches and phones to earbuds, smart rings, AR glasses, and body-worn sensors, creates new opportunities for sensing human motion in the wild, thus developing context-aware AI systems [@bulling2014tutorial; @haresamudram2025past; @cai2025towards]. Yet sensing motion is not the same as understanding it. Inertial measurement unit (IMU) signals are semantically ambiguous: similar inertial patterns can arise from different activities depending on who is moving, how, and in what context. Resolving this ambiguity requires knowledge beyond closed-set activity labels. Language provides a natural source of such knowledge, as it is grounded in human descriptions of everyday behavior and supports compositional, open-ended semantics. Connecting wearable sensing to language therefore helps models interpret motion in terms that generalize beyond fixed labels, making it central to generalist wearable motion understanding. However, wearable IMU signals are tightly coupled to how and where the device is worn, making robust modeling difficult across sensing setups [@cai2025towards; @kunze2014sensor; @chang2020systematic; @sztyler2016onbody]. A wrist-worn watch emphasizes arm motion, glasses capture head motion, and a phone in a pocket measures dynamics coupled to the torso and legs, even when the underlying activity is the same. Small changes in mounting position or orientation within a body part can further alter the measured acceleration and angular velocity, while device hardware and sampling introduce additional shifts [@stisen2015smart; @hong2024crosshar]. Consequently, models trained for one setup often struggle to transfer to another setup across users, devices, and datasets. This setup dependence, combined with the challenges of grounding IMU in language, makes building a broadly useful wearable motion model difficult along three coupled axes [@haresamudram2025past; @cai2025towards; @bian2026foundation]. `\ding{182}`{=latex} **Data and Supervision Scarcity**: Real IMU data is difficult to collect at scale. It remains fragmented across body placements, device hardware, sampling rates, and datasets, and supervision is often limited to a small closed set of coarse activity labels rather than rich descriptions of how motion unfolds. `\ding{183}`{=latex} **Limited Realism of Synthetic Augmentation**: Synthetic or augmented sensor data must expand setup coverage without losing physical realism, but existing generation pipelines often remain tied to specific labels, activities, or sparse sensor placements. `\ding{184}`{=latex} **Modality Gap**: Connecting IMU to language requires bridging continuous, multi-sensor motion signals with discrete textual concepts, a modality gap that direct prompting or simple contrastive alignment does not fully resolve and that grows more severe as the number of sensors, channels, and body locations increases. `\autoref{fig:method_families}`{=latex} contextualizes these issues across classifier-based, contrastive, LLM-based, and synthetic-generation methods, which address parts of the problem but remain limited along different axes. These challenges motivate our key insight: **wearable setup variation is structured rather than arbitrary**. An IMU is attached to a body surface, and its signal is produced by the interaction of body motion, surface geometry, sensor orientation, and device response. This structure provides a geometry- and physics-based inductive bias for learning setup-robust body motion representations. We further argue that **language should be connected to wearable sensing through a compact motion representation rather than raw IMU streams**. Language models provide priors for open-vocabulary motion understanding, but raw numerical IMU tokens scale with sensors, channels, and time, while sensor location-specific tokenizers tie representations to fixed setups. Compact full-body tokens avoid both limitations, providing a stable interface between motion and language. ![Method families for wearable human motion understanding and radar plot comparing the performance of AnyMo with baselines across various tasks and capabilities.](sec/figure/Method_Families.png){#fig:method_families width="1\\linewidth"} With these insights, we introduce **AnyMo**, a geometry-aware framework for setup-agnostic human motion modeling in the wild. It aims to learn robust IMU representations across **Any** wearable setup for human **Mo**tion understanding. As shown in `\autoref{fig:method_families}`{=latex}, AnyMo connects **physics-grounded geometry-aware IMU simulation** with **geometry-aware setup-agnostic pre-training**, **full-body tokenization**, and **motion-language alignment**. The simulation stage generates dense geometry-aware IMU candidates over body-surface placements, providing a broad and plausible distribution of wearable locations and orientations. The pre-training stage constructs paired placement views and masked wearable observations, encouraging the encoder to learn setup-agnostic motion representations. The tokenizer converts multi-position IMU observations into compact full-body motion tokens, aligned with an LLM for open-vocabulary recognition, cross-modal retrieval, and motion captioning. To validate AnyMo, we benchmark it against state-of-the-art methods on three complementary tasks: zero-shot activity recognition, unseen cross-modal retrieval, and wearable IMU motion captioning. For zero-shot recognition, we use 14 completely unseen downstream datasets spanning classic HAR benchmarks and in-the-wild settings. Retrieval and captioning are evaluated under sim-to-real transfer on unseen Nymeria subjects and out-of-domain (OOD) zero-shot transfer to EgoExo4D. Across all tasks, AnyMo shows significant gains over baselines, establishing it as a generalist model for wearable motion understanding. Our contributions are as follows: `\ding{182}`{=latex} **We propose physics-grounded, geometry-aware IMU simulation** over dense body-surface placements, providing diverse and plausible synthetic signals that bridge synthetic pre-training and real wearable IMU. `\ding{183}`{=latex} **We develop a setup-agnostic representation learning method and full-body IMU tokenization**, aligning motion across synthetic placement views and mapping multi-position IMU into full-body motion tokens. `\ding{184}`{=latex} **We introduce an IMU-language generalist model** that supports a variety of wearable motion understanding tasks. # Related Works **Wearable Motion Representations and Setup Transfer.** Recent wearable motion models improve generalization through synthetic pretraining [@zhang2024unimts; @leng2024imugpt2; @leng2023generating; @miao2026wonderwall; @wei2025one], large-scale self-supervision or cross dataset adaptation [@xu2025relcon; @hong2024crosshar], and tokenization [@zhang2025mopformer]. Other works study setup variation via cross-location transfer [@fortes2022learning], simulated body-surface placement analysis [@ray2025w2w], or coordinate-conditioned flexible placement [@zhou2025imucoco]; however, they remain mainly recognition- or pose-centered and do not combine dense local surface-frame simulation, setup-agnostic pretraining, full-body tokenization, and motion-language generation. **Sensor-Language and Cross-Modal Grounding.** Sensor-language and multimodal methods connect IMU or broader human-sensing signals to language, LLM reasoning, shared embedding spaces, or cross-modal supervision [@zhang2025sensorlm; @luo2026toward; @li2025sensorllm; @ji2024hargpt; @asif2025llasa; @wang2024ubiphysio; @su2025imuzero; @moon2023imu2clip; @girdhar2023imagebind; @zhou2026holollm; @leng2023generating; @leng2024imugpt2; @chen2025comodo; @tong2022zero; @li2025zara; @hong2025egolm; @wang2025ego4o; @jiang2023motiongpt; @nguyen2026mobind; @kinfu2025motionbind]. Embedding and instruction-tuning methods [@yu2025cafe; @lee2025nvembed; @muennighoff2025generative] further motivate joint retrieval and generation training, but these works do not directly address sparse, setup-dependent wearable IMU through geometry-aware full-body tokenization. **Self-Supervised Skeleton and Graph Motion Pretraining.** Skeleton and graph self-supervised methods [@yan2023skeletonmae; @mao2023masked; @huang2023graph; @lin2023actionlet; @hou2023graphmae2; @li20213d] motivate our use of masked motion modeling, graph encoders, and cross-view consistency, but these works focus on pose or skeleton observations rather than sparse wearable inertial signals. # Methodology {#sec:methodology} Building on the observation that wearable setup variation is structured, AnyMo targets wearable IMU motion understanding under variable sensing setups. We follow the Nymeria [@ma2024nymeria] body model and organize motion over $N_{\mathrm{seg}}=23$ anatomical segments. We denote an IMU window as $\mathbf{x}\in\mathbb{R}^{T\times 6}$, where the last dimension contains three-axis acceleration and three-axis angular velocity. The same motion can therefore yield different IMU windows across different wearable setups, whereas a real device typically provides only a partial observation of the body. AnyMo aims to learn a representation that absorbs such partial, setup-specific IMU windows and preserves motion information that is useful across sensing setups and language-based tasks. `\autoref{fig:method_families}`{=latex} illustrates the proposed pipeline with three key enablers: geometry-aware IMU simulation (`\autoref{sec:method_simulation}`{=latex}), setup-agnostic representation learning (`\autoref{sec:method_pretraining}`{=latex}), and full-body IMU tokenization with motion-language alignment (`\autoref{sec:method_motion_language}`{=latex}). ## Physics-Grounded Geometry-Aware Motion Simulation {#sec:method_simulation} Motion skeleton and mesh data describe human body motion through segment positions, orientations, and posed body-surface geometry over time [@ma2024nymeria]. Wearable IMUs, however, measure local acceleration and angular velocity rather than body pose directly. We synthesize wearable IMU windows by applying wearable IMU motion equations [@oishi2025wimusim] to synchronized Nymeria body motion. Unlike joint-centric simulation [@zhang2024unimts; @leng2023generating; @leng2024imugpt2; @wei2025one; @miao2026wonderwall], our goal is to model plausible wearable locations on the body surface, together with their local sensor frames and device noise. For each anatomical segment $i$, let $\mathcal{V}_i$ denote the selected candidate surface vertices on the Nymeria template body mesh. We compute a segment centroid $\mathbf{c}_i$ as the weighted average of the selected template vertices in $\mathcal{V}_i$. ```{=latex} \begin{wrapfigure}{r}{0.375\linewidth} % \includegraphics[width=\linewidth]{sec/figure/Simulation.pdf} \caption{Physics-grounded geometry-aware motion simulation.} \label{fig:simulation} % \end{wrapfigure} ``` To define a consistent in-surface direction, we set an anatomical axis $\mathbf{u}_i$ from $\mathbf{c}_i$ toward the centroid of its nearest available child segment in the body kinematic tree, or along the opposite direction from its nearest available parent when no child segment is available. For each vertex $v\in\mathcal{V}_i$, we compute a surface normal $\mathbf{n}_{i,v}$ from the template mesh faces. The normal defines a local tangent plane. We choose the tangent direction $\mathbf{t}_{i,v}$ by projecting $\mathbf{u}_i$ onto this plane, set the binormal $\mathbf{b}_{i,v}=\mathbf{n}_{i,v}\times\mathbf{t}_{i,v}$, and form a right-handed surface-based sensor frame $\mathbf{R}^{\mathrm{surf}}_{i,v}=[\mathbf{t}_{i,v},\mathbf{b}_{i,v},\mathbf{n}_{i,v}]$: $$\mathbf{t}_{i,v} = \left(\mathbf{u}_i-(\mathbf{u}_i^\top\mathbf{n}_{i,v})\mathbf{n}_{i,v}\right) /\left\|\mathbf{u}_i-(\mathbf{u}_i^\top\mathbf{n}_{i,v})\mathbf{n}_{i,v}\right\|_2. \label{eq:surface_tangent}$$ Let $\mathbf{p}_i(t)$, $\mathbf{R}_i(t)$, and $\mathbf{m}_v(t)$ denote the global segment position, global segment orientation, and posed mesh vertex position. We estimate the local virtual sensor offset by $\mathbf{r}_{i,v}=\frac{1}{T}\sum_{t=1}^{T}\mathbf{R}_i(t)^\top(\mathbf{m}_v(t)-\mathbf{p}_i(t))$. To account for mounting orientation variation during synthesis, we sample an in-plane rotation $\Delta\mathbf{R}_{i,v}$ around the surface normal and obtain the final local sensor frame $\widetilde{\mathbf{R}}^{\mathrm{surf}}_{i,v}=\Delta\mathbf{R}_{i,v}\mathbf{R}^{\mathrm{surf}}_{i,v}$. The virtual IMU trajectory is then defined by its global position $\mathbf{p}^{\mathrm{imu}}_{i,v}(t)=\mathbf{p}_i(t)+\mathbf{R}_i(t)\mathbf{r}_{i,v}$ and orientation $\mathbf{R}^{\mathrm{imu}}_{i,v}(t)=\mathbf{R}_i(t)\widetilde{\mathbf{R}}^{\mathrm{surf}}_{i,v}$. The accelerometer $\mathbf{a}_{i,v}(t)$ is obtained by transforming the second-order derivative of the virtual sensor position into the local sensor frame and removing gravity, while the gyroscope $\boldsymbol{\omega}_{i,v}(t)$ is computed from the temporal change of the virtual sensor orientation: $$\begin{aligned} \mathbf{a}_{i,v}(t) &= \mathbf{R}^{\mathrm{imu}}_{i,v}(t)^\top \left( d^2\mathbf{p}^{\mathrm{imu}}_{i,v}(t)/dt^2 -\mathbf{g} \right) + \boldsymbol{\eta}^{a}_{i,v}(t), \\ \boldsymbol{\omega}_{i,v}(t) &= \mathbf{R}^{\mathrm{imu}}_{i,v}(t)^\top \Omega\left(\mathbf{R}^{\mathrm{imu}}_{i,v}\right)(t) + \boldsymbol{\eta}^{\omega}_{i,v}(t). \end{aligned} \label{eq:sim_imu}$$ Here $\mathbf{g}$ denotes gravity, $\Omega(\cdot)$ maps an orientation trajectory to angular velocity, and $\boldsymbol{\eta}^{a}_{i,v}(t)$ and $\boldsymbol{\eta}^{\omega}_{i,v}(t)$ denote accelerometer and gyroscope noise. To reflect real device variability, we estimate two hardware-style noise priors from quiet windows of two real Nymeria IMU streams and randomly assign these priors to synthetic placements. The final synthetic IMU candidate for placement $(i,v)$ is $\mathbf{x}_{i,v}(t)=[a^{x}_{i,v}(t),a^{y}_{i,v}(t),a^{z}_{i,v}(t),\omega^{x}_{i,v}(t),\omega^{y}_{i,v}(t),\omega^{z}_{i,v}(t)]$. Collecting $\mathbf{x}_{i,v}$ over all selected vertices and anatomical segments yields a dense, geometry-aware distribution of wearable setups for pre-training. We evaluate the contribution of this geometry-aware simulation design in `\autoref{tab:ablation}`{=latex}. ## Geometry-Aware Setup-Agnostic Pre-Training {#sec:method_pretraining} The dense simulation in `\autoref{sec:method_simulation}`{=latex} provides multiple plausible IMU candidates for the same body motion, but downstream wearable inputs are sparse and setup-specific. Even within the same body segment, different surface placements and sensor orientations can produce different IMU windows. These setup variations are nevertheless organized by a fixed body topology: each synthetic IMU candidate is associated with one Nymeria anatomical segment, and segment motions are coupled through the body kinematic tree. We use this structure to represent a full-body IMU observation as a spatio-temporal graph, where node $i$ stores the IMU window sampled for segment $i$ and edges follow the Nymeria kinematic tree. Following spatial-temporal graph convolutional networks [@yan2018spatial], the graph encoder models temporal dynamics and cross-segment motion correlations while treating surface placement and mounting orientation as within-segment setup variation. This requires a representation that remains stable across synthetic setup changes while retaining temporal motion structure beyond coarse activity labels. As shown in `\autoref{fig:pretraining}`{=latex}, we sample paired synthetic placement views from these candidates to train a setup-agnostic graph encoder and a full-body IMU tokenizer. ![Details of (1) Geometry-Aware Pre-Training, (2) Full-Body IMU Tokenization and (3) Motion Language Model Pre-Training.](sec/figure/Pretraining.png){#fig:pretraining width="0.95\\linewidth"} **Masked Cross-View Predictive Contrastive Learning.** A natural choice is to contrast the two full graph views, following graph and skeleton contrastive representation learning [@huang2023graph; @lin2023actionlet; @li20213d]. However, full-view graph contrast can be satisfied by aligning complete body observations, so it does not teach the encoder to infer full-body motion from sparse wearable inputs. Masked modeling methods address sparsity by reconstructing masked graph or motion tokens [@yan2023skeletonmae; @mao2023masked; @hou2023graphmae2], but masked prediction alone does not explicitly separate different motion instances. Moreover, collapsing a motion window into a single clip-level embedding would remove the temporal structure needed by the tokenizer. We therefore design a masked cross-view predictive contrastive objective that combines sparse-to-full recovery, contrastive discrimination, and time-preserving sequence latents. Concretely, we predict the full-view latent of one synthetic setup from the masked observation of another setup, and contrast the prediction against other motion windows in the batch. We further analyze the importance of this learning objective through ablation and embedding visualization in `\autoref{tab:ablation}`{=latex} and `\autoref{fig:umap}`{=latex}. As illustrated in `\autoref{fig:pretraining}`{=latex}, for each motion window we construct two full graph views $A$ and $B$ by independently sampling candidate placements $v_i^A,v_i^B\in\mathcal{V}_i$ for every segment $i$. For each selected placement, we sample one local mounting rotation $\Delta\mathbf{R}^{A}_{i}$ or $\Delta\mathbf{R}^{B}_{i}$. The rotation combines in-plane rotation around the surface normal with a small tilt around the local tangent axes to approximate imperfect surface attachment. For an IMU candidate $\mathbf{x}_{i,v}(t)=[\mathbf{a}_{i,v}(t);\boldsymbol{\omega}_{i,v}(t)]$, rotation augmentation applies the same local rotation to acceleration and angular velocity, yielding paired full graph views $\mathbf{X}^{A},\mathbf{X}^{B}\in\mathbb{R}^{T\times N_{\mathrm{seg}}\times 6}$, where $\mathbf{X}^{A}_{t,i,:}=[\Delta\mathbf{R}^{A}_{i}\mathbf{a}_{i,v_i^A}(t);\Delta\mathbf{R}^{A}_{i}\boldsymbol{\omega}_{i,v_i^A}(t)]$ and $\mathbf{X}^{B}_{t,i,:}=[\Delta\mathbf{R}^{B}_{i}\mathbf{a}_{i,v_i^B}(t);\Delta\mathbf{R}^{B}_{i}\boldsymbol{\omega}_{i,v_i^B}(t)]$. We then create masked graph views by randomly keeping between one and five visible segment nodes and replacing all other segment nodes with a learnable mask token, as shown in `\autoref{fig:pretraining}`{=latex}. Let $\mathcal{M}^{A}$ and $\mathcal{M}^{B}$ denote the visible segment sets. The shared spatio-temporal graph encoder $\mathcal{G}_{\theta}$ produces node-level sequence features and averages over segment nodes to obtain time-preserving latents in $\mathbb{R}^{T'\times d}$, where $T'$ is the encoder output length and $d$ is the latent dimension. We write the full-view latents as $\mathbf{z}^{\mathrm{full}}_{A}=\mathcal{G}_{\theta}(\mathbf{X}^{A})$ and $\mathbf{z}^{\mathrm{full}}_{B}=\mathcal{G}_{\theta}(\mathbf{X}^{B})$, and the masked-view latents as $\mathbf{z}^{\mathrm{mask}}_{A}=\mathcal{G}_{\theta}(\operatorname{Mask}(\mathbf{X}^{A},\mathcal{M}^{A}))$ and $\mathbf{z}^{\mathrm{mask}}_{B}=\mathcal{G}_{\theta}(\operatorname{Mask}(\mathbf{X}^{B},\mathcal{M}^{B}))$. A temporal predictor $q_{\phi}$, implemented as a six-layer Transformer [@vaswani2017attention], predicts the opposite full-view latent: $\hat{\mathbf{z}}_{A\rightarrow B}=q_{\phi}(\mathbf{z}^{\mathrm{mask}}_{A})$ and $\hat{\mathbf{z}}_{B\rightarrow A}=q_{\phi}(\mathbf{z}^{\mathrm{mask}}_{B})$. We train the encoder and predictor with a cross-view predictive InfoNCE loss, using mean cosine similarity over time for predicted and target sequence latents $\hat{\mathbf{z}},\mathbf{z}\in\mathbb{R}^{T'\times d}$, defined as $s(\hat{\mathbf{z}},\mathbf{z})=\frac{1}{T'}\sum_{\ell=1}^{T'}\frac{\hat{\mathbf{z}}_{\ell}^{\top}\mathbf{z}_{\ell}}{\|\hat{\mathbf{z}}_{\ell}\|_2\|\mathbf{z}_{\ell}\|_2}$. For a minibatch of $N_{\mathrm{batch}}$ windows, the $A\rightarrow B$ loss is: $$\mathcal{L}_{A\rightarrow B} = -\frac{1}{N_{\mathrm{batch}}} \sum_{n=1}^{N_{\mathrm{batch}}} \log \frac{ \exp(s(\hat{\mathbf{z}}_{A\rightarrow B}^{(n)},\operatorname{sg}(\mathbf{z}_{B}^{\mathrm{full},(n)}))/\tau) }{ \sum_{m=1}^{N_{\mathrm{batch}}} \exp(s(\hat{\mathbf{z}}_{A\rightarrow B}^{(n)},\operatorname{sg}(\mathbf{z}_{B}^{\mathrm{full},(m)}))/\tau) }. \label{eq:mcvpcl_loss_ab}$$ The $B\rightarrow A$ loss is defined symmetrically, and the final objective is $\mathcal{L}_{\mathrm{MCVPCL}} = \mathcal{L}_{A\rightarrow B} + \mathcal{L}_{B\rightarrow A}$. Here $\operatorname{sg}(\cdot)$ denotes stop-gradient and $\tau$ is temperature, $(n)$ and $(m)$ index windows in the minibatch. **Full-Body IMU Tokenization.** Pre-training produces continuous, setup-stable sequence latents, while the motion-language model requires compact discrete inputs. As motivated in `\autoref{sec:introduction}`{=latex}, feeding raw IMU streams into an LLM is inefficient. To connect wearable motion with language models, we train a full-body IMU tokenizer that discretizes the frozen graph encoder latent into compact IMU tokens, following recent motion-language tokenizers based on product quantization [@hong2025egolm]. As shown in `\autoref{fig:pretraining}`{=latex}, with additional details in `\autoref{fig:tokenization_pretraining_detail}`{=latex}, we freeze $\mathcal{G}_{\theta}$ and train a product-quantized VAE tokenizer on the latent obtained from masked wearable observations $\mathbf{z}^{\mathrm{mask}}=\mathcal{G}_{\theta}(\operatorname{Mask}(\mathbf{X},\mathcal{M}))\in\mathbb{R}^{T'\times d}$. A projection maps each timestep latent $\mathbf{z}^{\mathrm{mask}}_{\ell}$ to a lower-dimensional projected latent $\bar{\mathbf{z}}_{\ell}\in\mathbb{R}^{\bar d}$. Let $P$ denote the number of product codebooks and $K$ denote the number of entries in each codebook. We evenly split $\bar{\mathbf{z}}_{\ell}$ into $P$ chunks $\{\bar{\mathbf{z}}_{\ell,j}\in\mathbb{R}^{\bar d/P}\}_{j=1}^{P}$, where $j$ indexes the product subspace. The $j$-th codebook is $\mathcal{E}_{j}=\{\mathbf{e}_{j,k}\in\mathbb{R}^{\bar d/P}\}_{k=1}^{K}$, where $\mathbf{e}_{j,k}$ is the $k$-th code vector in that codebook. Each chunk is quantized to its nearest code vector: $\kappa_{\ell,j}=\arg\min_{k\in\{1,\ldots,K\}}\|\bar{\mathbf{z}}_{\ell,j}-\mathbf{e}_{j,k}\|_2^2$, and the concatenated quantized latent $\mathbf{q}_{\ell}=[\mathbf{e}_{1,\kappa_{\ell,1}};\ldots;\mathbf{e}_{P,\kappa_{\ell,P}}]$, where $\kappa_{\ell,j}$ is the discrete code index for timestep $\ell$ and product subspace $j$, and $\mathbf{q}_{\ell}$ is the concatenated quantized latent. A temporal convolutional decoder takes quantized sequence $\mathbf{q}=[\mathbf{q}_{1},\ldots,\mathbf{q}_{T'}]$ and reconstructs $\tilde{\mathbf{z}}=D(\mathbf{q})$. We optimize the tokenizer with a reconstruction and commitment objective: $$\mathcal{L}_{\mathrm{com}} = \sum_{j=1}^{P} \frac{P}{T'\bar d} \sum_{\ell=1}^{T'} \left\| \bar{\mathbf{z}}_{\ell,j} - \operatorname{sg}(\mathbf{e}_{j,\kappa_{\ell,j}}) \right\|_2^2, \quad \mathcal{L}_{\mathrm{tok}} = \operatorname{SmoothL1}(\tilde{\mathbf{z}},\mathbf{z}^{\mathrm{mask}}) + \lambda_{\mathrm{com}}\mathcal{L}_{\mathrm{com}}. \label{eq:tokenizer_loss}$$ We update codebooks with exponential moving averages and refresh dead codes to maintain codebook usage. Finally, the IMU token sequence is formed by interleaving product-code indices over time, $\mathbf{s}=[\kappa_{1,1},\ldots,\kappa_{1,P},\kappa_{2,1},\ldots,\kappa_{T',P}]$. These discrete tokens preserve the temporal order of the setup-stable motion latent and serve as the IMU input tokens for motion-language alignment in `\autoref{sec:method_motion_language}`{=latex}. Since both $\mathcal{G}_{\theta}$ and the tokenizer operate temporally, variable-length IMU windows are handled by producing variable-length token sequences. ## Motion-Language Modeling {#sec:method_motion_language} The tokenizer in `\autoref{sec:method_pretraining}`{=latex} converts sparse wearable observations into discrete IMU token sequences, but these new tokens are not yet meaningful to a pretrained LLM. We therefore use motion language model pre-training to introduce the IMU vocabulary into the LLM and teach the model to understand wearable motion tokens. Multi-task contrastive instruction tuning aligns IMU-token prompts with language descriptions and activity-label prompts for retrieval, captioning, and zero-shot recognition. **Motion Language Model Pre-Training.** As shown in `\autoref{fig:pretraining}`{=latex}, with details in `\autoref{fig:tokenization_pretraining_detail}`{=latex}, motion language model pre-training adapts the LLM to the IMU token vocabulary. We extend the LLM vocabulary with IMU tokens and one code token for each entry in each product codebook. Rather than treating these tokens as unrelated new words, we use the learned tokenizer codebooks to give each IMU token a motion-aware embedding. Specifically, for each IMU code token associated with a codebook vector $\mathbf{e}_{j,k}$, a projector $g_{\rho}$ maps $\mathbf{e}_{j,k}$ into the LLM embedding space, and the corresponding input embedding is replaced by $g_{\rho}(\mathbf{e}_{j,k})$ during model execution. The same projected vectors are also used to initialize the corresponding rows in the LM head. We then continue causal LM pre-training on the interleaved IMU token sequence $\mathbf{s}$ using next-token cross-entropy loss $\mathcal{L}_{\mathrm{CE}}=\operatorname{CE}(p(\cdot\mid s_{20 classes) settings. For retrieval and captioning, we evaluate sim-to-real transfer on held-out Nymeria subjects and OOD zero-shot transfer to EgoExo4D. We compare against sensor-language multimodal methods [@girdhar2023imagebind; @moon2023imu2clip; @zhang2024unimts], synthetic IMU pre-training methods [@leng2023generating; @zhang2024unimts], wearable foundation model [@luo2026toward], and multimodal LLM baselines [@ji2024hargpt; @openai2025gpt54thinking; @googledeepmind2026gemma4modelcard]. We report Accuracy, macro-F1, and Recall@2 for recognition; Recall@K and MRR for retrieval; and BLEU, ROUGE-L, METEOR, and BERT-F1 for captioning. Full dataset statistics, preprocessing, prompt templates, and baseline details are provided in `\autoref{sec:experiment_details}`{=latex}, `\autoref{sec:downstream_evaluation_dataset}`{=latex}, and `\autoref{sec:baseline_details}`{=latex}. Separately, `\autoref{sec:anymo_bench}`{=latex} describes AnyMo Bench, an additional in-the-wild HAR benchmark built from our curated Nymeria activity labels for fine-grained unseen-subject and cross-device recognition. **Zero-shot Human Activity Recognition.** `\autoref{tab:zeroshot_har}`{=latex} summarizes zero-shot activity recognition on 14 completely unseen wearable datasets. AnyMo achieves the best average performance across all three metrics, with 35.7 Accuracy, 29.5 macro-F1, and 57.5 Recall@2, improving over the strongest average baseline by 11.7%, 11.6%, and 22.6%, respectively. The gains hold across controlled HAR benchmarks and in the wild datasets, suggesting that AnyMo learns transferable motion-language representations. ImageBind and IMU2CLIP provide sensor-language multimodal alignment and perform competitively on a few low-class datasets, but their performance drops as sensing setups and label spaces become more diverse. IMUGPT and UniMTS benefit from synthetic IMU motion pre-training, with UniMTS serving as the strongest prior baseline across the benchmark. AnyMo further improves over them through geometry-aware surface simulation and pre-training, and motion-language alignment. HARGPT and the Gemma 4 26B prompting baselines test whether substantially larger LLM priors alone can support zero-shot HAR: HARGPT uses role-play and step-by-step prompts over raw IMU readings, while Gemma 4 26B uses numerical IMU input or plots of IMU readings for multimodal understanding. The Gemma 4 26B prompting baselines occasionally perform well on individual datasets, but remain substantially below AnyMo on average, indicating that direct language or vision-language prompting, even at larger scale, is not sufficient for robust cross-setup wearable motion recognition. NormWear, despite pre-training on heterogeneous physiological and inertial signals including IMU, performs weakly in this benchmark. Suggesting that broad wearable-signal pre-training alone does not directly transfer to language-grounded open-vocabulary HAR across diverse unseen datasets and body placements. ```{=latex} \scriptsize ``` ```{=latex} \setlength{\tabcolsep}{2.2pt} ``` ```{=latex} \renewcommand{\arraystretch}{1} ``` ```{=latex} \resizebox{\linewidth}{!}{% \begin{tabular}{c|l|*{4}{c}|*{4}{c}|*{4}{c}|*{4}{c}} \toprule Dataset & Nymeria Held-out & EgoExo4D Zero-shot \\ \cmidrule(lr){1-2}\cmidrule(lr){3-10}\cmidrule(lr){11-18} \multirow{2}{*}{Task} & \multirow{2}{*}{Method} & 100 Samples & All Samples & 100 Samples & All Samples \\ \cmidrule(lr){3-6}\cmidrule(lr){7-10}\cmidrule(lr){11-14}\cmidrule(lr){15-18} & & R@1 & R@5 & R@10 & MRR & R@1 & R@5 & R@10 & MRR & R@1 & R@5 & R@10 & MRR & R@1 & R@5 & R@10 & MRR \\ \midrule \shortstack{IMU\\$\downarrow$\\Text} & ImageBind & 0.0 & 6.0 & 14.0 & 5.0 & 0.1 & 0.2 & 0.3 & 0.3 & 1.0 & 5.0 & 8.0 & 4.6 & 0.1 & 0.2 & 0.3 & 0.3 \\ & IMU2CLIP & 1.0 & 6.0 & 12.0 & 5.5 & 0.0 & 0.1 & 0.3 & 0.3 & \underline{2.0} & \textbf{10.0} & \underline{23.0} & \underline{8.2} & 0.0 & 0.3 & 0.5 & 0.4 \\ & UniMTS & \underline{4.0} & \underline{12.0} & \underline{23.0} & \underline{10.0} & \underline{0.2} & \underline{0.9} & \underline{1.6} & \underline{0.9} & 1.0 & 9.0 & 16.0 & 6.3 & \underline{0.1} & \underline{0.6} & \underline{1.3} & \underline{0.7} \\ & GPT-5.4 Mini & 1.0 & 7.0 & 11.0 & 4.4 & -- & -- & -- & -- & 1.0 & 6.0 & 10.0 & 3.7 & -- & -- & -- & -- \\ & Gemma 4 26B & 2.0 & 9.0 & 16.0 & 6.1 & -- & -- & -- & -- & 2.0 & 4.0 & 12.0 & 4.6 & -- & -- & -- & -- \\ & \textbf{AnyMo} & \textbf{28.0} & \textbf{63.0} & \textbf{77.0} & \textbf{44.6} & \textbf{2.3} & \textbf{9.5} & \textbf{15.4} & \textbf{7.0} & \textbf{2.0} & \underline{9.0} & \textbf{27.0} & \textbf{9.5} & \textbf{0.2} & \textbf{0.7} & \textbf{1.4} & \textbf{0.8} \\ \midrule \shortstack{Text\\$\downarrow$\\IMU} & ImageBind & \underline{1.0} & \underline{8.0} & \underline{14.0} & \underline{6.7} & \underline{0.1} & \underline{0.2} & 0.3 & 0.3 & \underline{2.0} & 3.0 & 7.0 & 5.1 & 0.0 & 0.0 & 0.2 & 0.2 \\ & IMU2CLIP & 0.0 & 6.0 & 14.0 & 5.0 & 0.1 & 0.2 & 0.3 & \underline{0.3} & 1.0 & \underline{9.0} & \underline{17.0} & \underline{7.7} & \textbf{0.1} & \textbf{0.3} & \underline{0.5} & \underline{0.4} \\ & UniMTS & 1.0 & 6.0 & 12.0 & 5.5 & 0.1 & 0.2 & \underline{0.4} & 0.3 & 1.0 & 5.0 & 10.0 & 5.3 & 0.0 & 0.1 & 0.3 & 0.2 \\ & GPT-5.4 Mini & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- \\ & Gemma 4 26B & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- & -- \\ & \textbf{AnyMo} & \textbf{33.0} & \textbf{60.0} & \textbf{79.0} & \textbf{46.7} & \textbf{3.0} & \textbf{9.9} & \textbf{16.1} & \textbf{7.5} & \textbf{3.0} & \textbf{10.0} & \textbf{23.0} & \textbf{9.9} & \underline{0.0} & \underline{0.3} & \textbf{0.6} & \textbf{0.4} \\ \bottomrule \end{tabular} } ``` **Cross-Modal Retrieval.** `\autoref{tab:unseen_cross_modal_retrieval}`{=latex} evaluates bidirectional retrieval between sparse IMU windows and motion narrations. The Nymeria held-out split is designed as a synthetic-to-real transfer test: AnyMo is trained without the held-out subjects using synthetic IMU candidates generated from body mesh/skeleton motion, while all methods are evaluated on the same real IMU streams from held-out subjects. ImageBind and IMU2CLIP reflect large-scale real sensor-language pre-training, UniMTS reflects an alternative synthetic IMU pre-training strategy, and GPT-5.4 Mini/Gemma 4 26B test whether much larger state-of-the-art LLMs can solve retrieval through prompting. AnyMo substantially outperforms all baselines on Nymeria held-out, improving 100-sample IMU$\rightarrow$Text MRR from 10.0 to 44.6 and Text$\rightarrow$IMU MRR from 6.7 to 46.7, while remaining clearly ahead in the harder all-sample setting. EgoExo4D further evaluates OOD zero-shot transfer. Although all methods degrade, AnyMo achieves the best or second-best performance on most metrics. ![Qualitative Results of Wearable IMU Motion Caption Generation. We use green to highlight correct parts and red for mistakes.](sec/figure/Caption_Qualitative.png){#fig:qualitative_caption width="\\linewidth"} **Wearable IMU Motion Captioning.** `\autoref{tab:wearable_imu_caption_generation}`{=latex} evaluates AnyMo's generative motion-language capability. Under the same Nymeria held-out and EgoExo4D zero-shot settings, AnyMo outperforms GPT-5.4 Mini and Gemma 4 26B across captioning metrics. On Nymeria held-out, AnyMo substantially improves over the strongest prompting baseline, increasing ROUGE-L from 15.7 to 31.1 and BERT-F1 from 57.3 to 69.7. These gains show that the learned IMU tokens retain motion semantics that can be decoded into natural-language descriptions. The advantage persists under EgoExo4D zero-shot transfer, where AnyMo achieves 20.7 BLEU-1, 30.3 METEOR, and 67.1 BERT-F1 despite OOD motion distributions. This suggests that AnyMo provides stronger generative modeling than direct LLM prompting over IMU observations. `\autoref{fig:qualitative_caption}`{=latex} provides qualitative examples. The prompting baselines often describe generic posture changes or local signal fluctuations, such as standing, head motion, or hand movement, while missing the full activity. In contrast, AnyMo produces captions that better preserve the action-level semantics, including walking direction changes in Nymeria and cabinet-closing interactions in EgoExo4D. These results suggest that the IMU tokenizer and motion-language instruction tuning enable open-ended motion description from wearable signals. ```{=latex} \scriptsize ``` ```{=latex} \setlength{\tabcolsep}{3.2pt} ``` ```{=latex} \renewcommand{\arraystretch}{1} ``` ::: {#tab:wearable_imu_caption_generation} Dataset Nymeria Held-out EgoExo4D Zero-shot ----------------------------------------------------------------- ------------------ -------------------- ---------- ---------- ---------- ---------- --------- ---------- ---------- ---------- 1-1`\cmidrule`{=latex}(lr)2-6`\cmidrule`{=latex}(lr)7-11 Method BLEU-1 BLEU-4 ROUGE-L METEOR BERT-F1 BLEU-1 BLEU-4 ROUGE-L METEOR BERT-F1 GPT-5.4 Mini 19.2 0.3 15.7 25.0 57.3 12.6 0.0 15.5 23.8 56.5 Gemma 4 26B 16.2 0.0 13.6 21.5 56.5 3.5 0.0 4.6 6.4 55.1 **AnyMo** **25.0** **6.5** **31.1** **33.5** **69.7** **20.7** **0.4** **19.7** **30.3** **67.1** : Unseen and zero-shot IMU motion caption generation results on Nymeria and EgoExo4D. ::: ![UMAP visualization of paired real and synthetic IMU embeddings for ten activity categories.](sec/figure/umap.png){#fig:umap width="\\linewidth"} **Ablation.** `\autoref{tab:ablation}`{=latex} ablates each component (see `\autoref{sec:ablation_implementation_details}`{=latex} for detailed implementation). Removing geometry-aware simulation causes the largest degradation, reducing performance from 35.7/29.5/57.5 to 8.4/3.8/16.3 in Acc/F1/R@2. Replacing the masked cross-view predictive contrastive objective also severely hurts performance. These results indicate that synthetic IMU generation alone is insufficient: the model needs both realistic surface-aware setup variation and a sparse-to-full pre-training objective to transfer from synthetic full-body candidates to real sparse wearable inputs. `\autoref{fig:umap}`{=latex} further visualizes this effect. For 10 activity categories, we randomly sample 100 real and 100 synthetic windows per activity and plot their embeddings with UMAP [@mcinnes2018umap]. ```{=latex} \begin{wraptable}{r}{0.45\textwidth} \caption{Ablation study.} \label{tab:ablation} \scriptsize \setlength{\tabcolsep}{3.2pt} \renewcommand{\arraystretch}{1} % \setlength{\aboverulesep}{0.25ex} % \setlength{\belowrulesep}{0.25ex} % \setlength{\cmidrulesep}{0.15ex} \begin{tabular}{lccc} \toprule Variant & Acc & F1 & R@2 \\ \midrule \textbf{AnyMo} & \underline{35.7} & \textbf{29.5} & \textbf{57.5} \\ \rowcolor{gray!12} \emph{Simulation and encoder pretraining} \\ w/o geometry-aware simulation & 8.4 & 3.8 & 16.3 \\ w/o masked cross-view pred. contrastive & 10.5 & 4.5 & 19.0 \\ \rowcolor{gray!12} \emph{Motion-language contrastive} \\ w/o label contrastive & 33.3 & 27.1 & 51.4 \\ w/o narration contrastive & 32.6 & 23.8 & 50.1 \\ w/o all contrastive & 23.5 & 15.8 & 40.3 \\ \rowcolor{gray!12} \emph{LM instruction tuning} \\ w/o MCQ instruction tuning & \textbf{36.0} & \underline{29.4} & \underline{54.6} \\ \bottomrule \end{tabular} % \end{wraptable} ``` AnyMo forms more discriminative activity clusters while aligning real and synthetic samples of the same activity. Without geometry-aware simulation, real and synthetic embeddings separate by domain, while removing masked cross-view predictive contrastive learning reduces cluster coherence, highlighting the importance of both components for synthetic-to-real alignment. The motion-language losses are also important. Removing label or narration contrastive tuning degrades Acc, F1, and R@2, showing that both label-level supervision and free-form narration alignment support zero-shot recognition. Removing all contrastive tuning further degrades performance, confirming that next-token instruction tuning alone is insufficient for discriminative open-vocabulary recognition. Finally, removing MCQ instruction tuning slightly increases Acc but reduces F1 and R@2, suggesting that the MCQ branch mainly improves balanced recognition and candidate ranking. # Conclusion {#sec:conclusion} We presented AnyMo, a geometry-aware setup-agnostic framework for wearable IMU motion understanding under variable sensing setups. AnyMo treats wearable setup variation as structured body-surface variation, synthesizes dense physics-grounded IMU candidates from human mesh motion, learns setup-agnostic full-body motion representations, and connects them to language models through compact full-body IMU tokens. Across zero-shot activity recognition, bidirectional IMU-text retrieval, and wearable motion caption generation, AnyMo shows consistent gains on unseen datasets, held-out real Nymeria IMU streams, and out-of-domain EgoExo4D transfer. These results suggest that combining geometry-aware wearable simulation with motion-language modeling is a promising path toward generalist motion understanding from sparse wearable sensors. ```{=latex} \begin{ack} This research includes computations using the Wolfpack computational cluster, supported by the School of Computer Science and Engineering at UNSW Sydney. We also acknowledge support from the ARC Centre of Excellence for Automated Decision-Making and Society (CE200100005). \end{ack} ``` ```{=latex} \bibliographystyle{plainnat} ``` ```{=latex} \newpage ``` ```{=latex} \appendix ``` # Experiments and Implementation Details {#sec:experiment_details} ## Training Data and Window Construction {#sec:training_data_window_construction} All AnyMo training stages use Nymeria [@ma2024nymeria] as the only source of motion-text supervision. We first synchronize Nymeria body motion, mesh vertices, atomic-action annotations, and real IMU streams to a common 60 Hz timeline. Most exported instruction-tuning rows correspond to 5 s windows: after removing IMU boundary tokens, 90.1% of rows contain 150 IMU code tokens, which corresponds to $T=300$ frames because the original frame count is twice the IMU code-token count. The remaining text-aligned rows keep their native atomic-action durations, so $T$ can be shorter or longer. Each frame contains six IMU channels (three-axis acceleration and three-axis angular velocity), and each window is represented on a fixed 23-node body graph following the Xsens kinematic tree: pelvis, spine, neck/head, both shoulders/arms/hands, and both legs/feet/toes. The graph representation has shape $6 \times T \times 23 \times 1$. For sparse wearable observations, only the nodes corresponding to the visible sensors are exposed to the model; the other graph nodes are replaced by the learned mask token in the encoder. Before applying the held-out-subject protocol, the text-aligned Nymeria export contains 828 recording samples and 168,295 windows. We choose the held-out subjects so that the held-out split remains subject-disjoint while still covering all 20 original Nymeria scenarios. Specifically, we reserve five subjects (`alec_meza`, `bradley_herman`, `dominique_frye`, `justin_ramirez`, and `kyle_parker`), whose held-out recordings collectively cover the 20 scenarios, and exclude these recordings from the text-aligned token and instruction-tuning exports. The resulting subject-disjoint IMU-token pre-training corpus contains 808 Nymeria recording samples and 164,387 text-aligned windows. The instruction-tuning export contains 159,098 labeled windows and expands them into 986,322 language-model instruction rows, evenly split between narration and activity multiple-choice tasks. The same subject-disjoint export provides 986,322 paired IMU-text contrastive rows, using the original atomic-action narration and conservative augmented narrations as positive texts. ## Synthetic IMU Generation {#sec:synthetic_imu_generation_details} The synthetic pre-training signals are generated from Nymeria's anatomically grounded human model, using its mesh and skeleton motion. For every body segment, we build a surface template from selected mesh vertices. At each candidate vertex, we estimate a local right-handed sensor frame from the surface normal, an anatomical tangent direction, and the corresponding binormal. The local offset between the posed mesh vertex and the segment coordinate frame is then tracked through the motion sequence, producing a dense set of geometry-aware candidate IMU placements. We define our surface selection from the template-mesh skinning weights. For each mesh vertex, we sort the influencing skeleton joints by skinning weight and assign the vertex to a body segment if one of that segment's joints appears among the top two nonzero influences. This is a compromise between two less useful extremes: a strict top-1 assignment can make shared boundary vertices disappear from nearby segments, while treating every positive skinning weight as a candidate can spread each segment over overly broad and weakly related surface regions. The resulting selected vertex counts are shown in `\autoref{tab:selected_surface_vertices}`{=latex}; in total, the exported synthetic archive contains 831 eligible Nymeria samples, each with 2,374 candidate placements across the 23 body segments. ::: {#tab:selected_surface_vertices} Segment \# Vertices Segment \# Vertices Segment \# Vertices ----------------- ------------- ----------------- ------------- ---------------- ------------- Pelvis 108 T8 105 Right Hand 145 L5 59 Neck 44 Left Shoulder 45 L3 35 Head 290 Left Upper Arm 90 T12 90 Right Shoulder 42 Left Forearm 120 Right Upper Arm 90 Left Hand 145 Left Upper Leg 106 Right Forearm 127 Right Upper Leg 108 Left Lower Leg 122 Right Lower Leg 122 Right Foot 125 Left Foot 112 Right Toe 72 Left Toe 72 : Selected surface vertices per body segment for synthetic IMU placement. ::: During representation pre-training, each training window is sampled twice to form two full-body graph views. For each segment in each view, one candidate placement is selected independently. We further apply surface rotation augmentation, with an in-plane rotation range of $\pm 180^\circ$ and a small tilt range of $\pm 10^\circ$ during training. Acceleration and angular velocity are computed from the simulated rigid-body trajectory in the candidate sensor frame. Real Nymeria IMU streams from the head and wrists are not used as training targets; they are used to estimate device-noise priors and to define the held-out sim-to-real evaluation protocol. We estimate this noise prior from the two Nymeria device streams and three real wearable sites to make the synthetic IMU signals more realistic. ![More details of Masked IMU Tokenization and Motion Language Model Pre-Training.](sec/figure/Tokenization_Pretraining_Detail.png){#fig:tokenization_pretraining_detail width="\\linewidth"} ## Representation Pre-Training and Tokenization {#sec:representation_pretraining_details} As illustrated in `\autoref{fig:tokenization_pretraining_detail}`{=latex}, the IMU encoder is an ST-GCN [@yan2018spatial] over the 23-node body graph. It uses ten spatio-temporal graph convolution blocks with temporal kernel size 9, channel widths increasing from 64 to 128 to 256, and two temporal stride-2 stages. Thus a 300-frame input window is encoded as a time-preserving latent sequence of length about 75, with latent dimension 256. Pre-training samples two independent full-body graph views from the synthetic candidate set and two corresponding sparse views. For each sparse view, a random number of visible nodes between 1 and 5 is retained, and all remaining nodes are masked. A six-layer Transformer [@vaswani2017attention] predictor maps the sparse-view latent sequence to the opposite full-view target sequence. We train with the symmetric predictive InfoNCE objective, temperature 0.1, and stop-gradient targets. Unless otherwise noted, the ST-GCN optimizer is AdamW with learning rate $3\times10^{-4}$, batch size 64, and 10 epochs. The ST-GCN encoder pre-training is run on a single NVIDIA L40S GPU. ::: {#tab:pqvae_tokenizer_diagnostics} Metric Codebook 0 Codebook 1 ------------------------------- --------------- --------------- Used codes 2,033 / 2,048 2,046 / 2,048 Usage rate 99.27% 99.90% Dead-code ratio 0.73% 0.10% Perplexity 1,285.7 1,098.1 Top-1 code mass 1.46% 1.11% Top-10 code mass 5.17% 4.96% Exact sequence collision rate 0.61% : Final PQ-VAE tokenizer diagnostics for the exported IMU tokenizer. Perplexity and top-$k$ code mass summarize codebook usage, while exact sequence collision is measured after interleaving the two codebook streams. ::: After ST-GCN pre-training, we freeze the encoder and train a product-quantized VAE on masked sparse-view latent sequences. The tokenizer uses two codebooks with 2,048 entries each, a 128-dimensional bottleneck, and 64-dimensional code vectors per codebook. The product-quantizer codebooks are updated with exponential-moving-average (EMA) statistics using decay 0.99. To prevent unused IMU tokens from persisting, we apply dead-code refresh after each EMA update: within each codebook, entries whose accumulated usage falls below 20% of the average code usage are replaced by latent vectors sampled from the current mini-batch. The decoder reconstructs the frozen ST-GCN latent sequence with a SmoothL1 reconstruction loss plus the standard commitment loss. At export time, the two codebook indices are interleaved over time. Consequently, a 5 s Nymeria window normally yields 75 latent steps and 150 IMU code tokens, plus boundary tokens. We add 4,096 IMU code tokens and IMU boundary tokens to the LLM tokenizer, while keeping the learned codebook vectors as the continuous representation behind those discrete token IDs. We report final tokenizer diagnostics in `\autoref{tab:pqvae_tokenizer_diagnostics}`{=latex}, including codebook usage, concentration statistics, and exact sequence collisions. These diagnostics check the tokenizer at two levels. At the codebook level, the near-complete usage rates and low dead-code ratios show that the product quantizer does not collapse to a small subset of entries. The perplexities, 1,285.7 and 1,098.1 out of 2,048 entries, indicate broad but non-uniform code usage, while the low top-1 and top-10 masses show that no small group of codes dominates the assignment distribution. At the sequence level, the exact sequence collision rate is computed after interleaving the two codebook streams over time; the 0.61% collision rate indicates that almost all tokenized IMU windows receive distinct discrete token sequences. ## Motion-Language Training {#sec:motion_language_training_details} AnyMo uses Qwen2.5-0.5B [@qwen2025qwen25technicalreport] as the language backbone. We implement the motion-language pre-training stage with ms-swift [@zhao2025swift], using a custom AnyMo model registration that replaces IMU token embeddings with projected codebook vectors at runtime. All motion-language training stages are run on two NVIDIA L40S GPUs. The LLM is first adapted with an IMU-token language-modeling stage in which the assistant target consists of IMU token sequences exported from the frozen ST-GCN and tokenizer. The IMU code-token embeddings are not ordinary lookup rows: at runtime, each IMU token ID is mapped back to its product-quantizer code vector and projected into the Qwen hidden space by a two-layer MLP. The corresponding language-model head rows are initialized from the same projected code vectors. For this stage, we use full fine-tuning with `bf16`, a maximum sequence length of 1024, learning rate $10^{-4}$, batch size 16, and 3 epochs. Before instruction tuning, we augment the Nymeria atomic-action narrations with GPT-OSS-120B [@openai2025gptoss120bgptoss20b]. For each atomic-action text, we treat the original annotation as the ground-truth semantic anchor and generate five paraphrases that preserve the same action meaning while varying wording, conciseness, temporal emphasis, body-motion emphasis, and scene generality. We then run a GPT-OSS-120B self-verifier that checks whether the five variants preserve the atomic action, maintain the action order, avoid unobserved intent or invented content, and provide sufficient diversity. If the verifier rejects the whole set, we regenerate all five variants; if fewer than five variants fail, we repair only the rejected variants while keeping the accepted ones. The verifier is applied after each generation or repair step, with at most two additional regeneration/repair rounds, giving at most three verifier passes per atomic-action row. For the label-contrastive branch, we construct a closed activity-label vocabulary from the same atomic-action annotations. We first ask GPT-OSS-120B to assign a free-form activity label to every atomic action. Because these free-form labels can differ substantially in surface form, we normalize them by lowercasing, removing punctuation and extra spaces, and canonicalizing leading articles such as \`\`a", \`\`an", and \`\`the" before exact deduplication. We embed labels with Qwen3-Embedding-8B [@zhang2025qwen3textembedding] and cluster labels with frequency at least five using a cosine-similarity threshold of 0.85. For clusters containing two or more labels, GPT-OSS-120B merges semantically close labels into either one simple label or a slash-separated label when multiple equivalent phrasings should be retained. Human experts review roughly 1,000 candidate labels, remove ambiguous categories, and finalize AnyMo-180, a 180-class activity-label vocabulary. The resulting AnyMo-180 label corpus turns Nymeria into one of the largest fine-grained IMU-based HAR training corpora, providing an activity vocabulary that goes beyond the small closed label spaces common in wearable HAR and helps mitigate coarse activity labels and fragmented activity vocabularies [@haresamudram2025past; @cai2025towards; @bian2026foundation]. To label windows against AnyMo-180, we run GPT-5.4 nano twice as an enum-text classifier over the 180 classes. Rows with identical labels across the two runs are accepted directly. Disagreements are adjudicated by GPT-5.4 mini, and we retain only labels with a majority vote after adjudication. The two GPT-5.4 nano passes agree on 128,461 of 168,295 rows (76.33%); among the 39,834 disagreements, GPT-5.4 mini produces a majority label for 34,873 rows. After this filtering, 162,896 rows remain labeled, corresponding to 96.79% of the 168,295 candidate rows. We then perform a second class-wise audit over the curated label set to further improve label quality. For each candidate row, we use GPT-5.5 High to compare the assigned activity label against the current source-text annotation and the preceding and following source-text annotations to verify the local temporal context; inconsistent rows are relabeled or dropped under human-expert supervision. This final audit yields 158,138 labeled rows, corresponding to 93.96% of candidate rows. The class distribution is long-tailed, reflecting in-the-wild activity frequencies: per-class counts range from 18 to 19,696 rows, with a median of 342.5 and a mean of 878.54. The final label counts are reported in `\autoref{tab:final_activity_label_counts}`{=latex}. ```{=latex} \tiny ``` ```{=latex} \setlength{\tabcolsep}{3pt} ``` ```{=latex} \begin{tabular}{@{}p{0.18\textwidth}r p{0.18\textwidth}r p{0.18\textwidth}r p{0.18\textwidth}r@{}} \toprule Label & Count & Label & Count & Label & Count & Label & Count \\ \midrule walking / hiking & 19{,}696 & standing & 7{,}411 & sitting & 5{,}906 & play board games / cards / puzzle & 5{,}873 \\ placing items on a surface & 4{,}758 & picking up objects & 4{,}248 & slicing / chopping / cutting / dicing food & 3{,}765 & folding clothes / pants / towel / blanket & 3{,}752 \\ gesturing & 3{,}716 & dancing & 3{,}254 & kneeling & 3{,}130 & turning in place / turning / spinning & 2{,}584 \\ playing soccer / kicking ball & 2{,}522 & closing refrigerator / cabinet / drawer / closet / door & 2{,}493 & playing charades & 2{,}476 & playing badminton & 2{,}419 \\ eating & 2{,}323 & standing up and walking & 2{,}239 & walking while carrying objects & 2{,}141 & opening and closing refrigerator / cabinet / drawer / closet / door & 2{,}116 \\ using laptop & 2{,}058 & raising hand(s) / arm(s) & 1{,}987 & adjusting pillows / sofa pillows & 1{,}979 & pulling a drawer & 1{,}941 \\ side stepping exercise & 1{,}887 & opening refrigerator / cabinet / drawer / closet / door & 1{,}828 & yoga / stretching & 1{,}778 & placing items into a box / bag / container & 1{,}584 \\ watching tv & 1{,}581 & arranging decorations & 1{,}536 & closing refrigerator / cabinet / drawer / closet / door and walking & 1{,}470 & playing air hockey / playing table hockey / playing foosball & 1{,}433 \\ walking down stairs & 1{,}345 & walking up stairs & 1{,}311 & squatting & 1{,}253 & biking outdoors & 1{,}165 \\ writing & 1{,}146 & opening refrigerator / cabinet / drawer / closet / door and walking & 1{,}114 & throwing objects & 1{,}036 & lying & 976 \\ scooping food & 971 & making coffee & 910 & playing arcade game & 882 & drinking & 872 \\ drawing & 857 & marching in place & 822 & measuring appliance / furniture / door & 802 & stirring / mixing & 795 \\ hanging clothing & 780 & disposing trash & 779 & moving chair & 703 & spreading / unfolding blanket / clothes & 664 \\ twisting torso & 635 & scratching / touching face & 623 & adding food to container / pan & 608 & loading dishwasher / unloading dishwasher & 601 \\ adjusting blinds / adjusting curtains & 593 & rotating hands / arms & 586 & opening bag / package / pack / pouch & 576 & inspecting appliance / furniture & 569 \\ walking / stepping in place & 555 & cleaning tableware / kitchenware & 549 & stir-frying & 544 & lunges / lunging & 541 \\ playing table tennis & 539 & using phone & 538 & standing up & 498 & opening refrigerator / cabinet / drawer / closet and taking objects & 477 \\ adjusting stove / turning on stove / opening stove / turning stove knob & 457 & knee raises & 447 & sweeping & 445 & spreading sauce / cream / butter on bread / food & 444 \\ turning off / on light switch & 441 & turning on faucet & 441 & playing billiards & 425 & walking while picking up object & 418 \\ adjusting glasses & 414 & tiptoeing & 412 & opening a box / container & 410 & cleaning whiteboard & 404 \\ sitting down & 398 & poking food & 393 & folding / rolling / tearing / peeling tape & 381 & punching / shadowboxing & 377 \\ reading & 374 & seasoning food & 367 & walking with bike & 353 & running & 352 \\ turning / shaking / nodding head & 346 & looking under furniture & 345 & closing a box / container & 340 & jumping / hopping in place & 319 \\ looking around & 319 & playing catch & 314 & butt kicks & 312 & operating electric kettle & 306 \\ playing darts & 302 & stacking items & 283 & flipping food in pan & 279 & capping marker / bottle / jar & 268 \\ stationary biking / riding an exercise bike & 267 & opening bottle / jar & 264 & vacuuming / walking while vacuuming & 256 & washing hands & 250 \\ moving sofa / couch & 249 & cleaning a surface & 243 & jogging in place & 243 & standing foot rotation / ankle rotation & 243 \\ operating oven & 240 & pressing / mashing food in pan & 240 & pouring liquid & 234 & using microwave & 224 \\ peeling fruits / vegetable & 223 & moving table & 221 & alternating punches while stepping in place & 218 & foot tapping & 217 \\ inserting pillow into pillowcase & 216 & placing pan on stove & 216 & swinging on a swing & 215 & pulling blinds / pulling curtains & 208 \\ filling a container with liquid & 195 & organizing books / shelves & 193 & marching & 187 & front kicks & 182 \\ air writing & 169 & standing leg raise & 168 & drying hands & 161 & using washing machine & 154 \\ jogging & 142 & playing with rubik's cube & 140 & eating while playing board game & 137 & pouring food into container & 137 \\ plank / alternating plank & 130 & whisking / beating eggs & 126 & cracking egg & 116 & bicep curls & 115 \\ organizing kitchen items & 114 & stepper exercise / using stepper & 114 & moving bed & 111 & ab rollout exercise & 110 \\ organizing clothes / towels & 110 & untangling party decorations & 110 & kneading dough / shaping dough & 109 & adjusting clothing & 108 \\ making sandwich & 89 & seated leg raises & 89 & adjusting hair & 85 & catching objects & 83 \\ running in place & 83 & rolling a yoga ball & 82 & adjusting headset & 80 & playing with a pool noodle & 80 \\ mopping & 77 & using toaster & 76 & bowling & 72 & walking while drinking & 72 \\ checking smartwatch & 69 & opening trash bin & 69 & clapping & 68 & jumping rope & 68 \\ tearing paper towel / tearing tissue & 68 & pulling chair and sitting & 65 & wiping mouth with tissue / napkin / paper towel & 61 & removing seeds from food & 59 \\ tying shoe laces & 57 & walking while punching & 51 & playing toy & 50 & standing arm circles & 47 \\ washing fruit / vegetables & 46 & writing with foot & 45 & dumbbell lateral raises & 43 & setting up pool table & 42 \\ hanging decorations / frames & 39 & threading beads & 38 & standing elbow-to-knee crunch & 35 & sit-ups & 34 \\ sitting up & 34 & playing rock-paper-scissors & 33 & pruning flower & 21 & squeezing fruit & 18 \\ \bottomrule \end{tabular} ``` We then perform instruction tuning with both language-modeling and contrastive losses. The language-modeling branch contains two task families: IMU-to-narration generation and IMU-to-activity multiple choice. For each multiple-choice example, the correct activity is mixed with a fixed 35-choice candidate set sampled from the training activity pool. The contrastive branch encodes IMU prompts and text prompts with the same Qwen backbone, then applies branch-specific latent-attention poolers and projection heads before a symmetric IMU-text contrastive loss. The reported contrastive checkpoint uses an 8-token learnable soft prompt on the text/label branch. We train the contrastive instruction-tuning stage for 1 epoch with maximum sequence length 4096, learning rate $2\times10^{-5}$, contrastive temperature 0.05, language-modeling loss weight 1.0, and contrastive loss weight 2.0. For contrastive batches, we use a per-GPU batch size of 16 and 4 GradCache steps [@gao2021scaling]; with NVIDIA L40S GPUs, this gives an effective global contrastive batch size of $16 \times 4 \times 2 = 128$. ## Zero-Shot Recognition Evaluation {#sec:zero_shot_recognition_eval_details} Downstream HAR datasets are used only for evaluation. All dataset adapters convert accelerometer channels to $\mathrm{m/s^2}$ and gyroscope channels to $\mathrm{rad/s}$ when the source units are known, resample windows to 60 Hz, and map each physical device location to the closest node(s) in the 23-node body graph. For head-mounted video datasets such as Ego4D, EgoExo4D, and MMEA, the visible node is `Head`. For wrist, arm, waist, torso, leg, and foot sensors, the adapters expose the corresponding upper-limb, spine, or lower-limb graph nodes. Each evaluation example is tokenized by the frozen ST-GCN and tokenizer, then scored without updating AnyMo. For recognition, we encode the IMU prompt and each candidate activity label prompt, compute cosine similarities in the learned contrastive space, and rank all labels from the dataset's label vocabulary. Accuracy is computed from the top-ranked label, macro-F1 is computed over predicted labels, and Recall@2 is positive if the ground-truth label appears in the top two ranked labels. The label vocabulary is dataset-specific and includes all activities for that benchmark split, so the evaluation does not require dataset-specific classifier heads. ## Retrieval and Captioning Evaluation {#sec:retrieval_captioning_eval_details} For sim-to-real evaluation on Nymeria, we export held-out examples from the five reserved subjects listed above. This produces 20 held-out recording samples and 3,908 text-aligned windows for both retrieval and captioning. For OOD evaluation on EgoExo4D, we construct a balanced 4,000-window atomic-action subset with 500 examples from each of eight parent tasks: rock climbing, basketball, dance, cooking, health, bike repair, soccer, and music. These windows are sampled from EgoExo4D's 200 Hz head-IMU streams, resampled to 60 Hz, and tokenized with the same frozen AnyMo tokenizer. Retrieval uses the same prompt templates as contrastive instruction tuning. For IMU-to-text retrieval, each IMU embedding is compared against the pool of primary reference narrations; for text-to-IMU retrieval, each text embedding is compared against the pool of IMU embeddings. We report Recall@1, Recall@5, Recall@10, and MRR in both directions. Captioning uses the IMU-to-narration prompt and greedy decoding with a maximum of 64 generated tokens. Generated captions are evaluated against the primary atomic-action narration using BLEU, ROUGE-L, METEOR, and BERT-F1. For BERT-F1, we compute it with the `deberta-xlarge-mnli` [@he2021deberta] [^2]. ## Capability Radar Details {#sec:capability_radar_details} `\autoref{fig:method_families}`{=latex} summarizes the main results with nine radar axes drawn from the three evaluation groups. Exact Activity Match, Balanced Recognition, and Top-2 Activity Recall correspond to the average zero-shot HAR Accuracy, macro-F1, and Recall@2 across the 14 downstream recognition datasets. IMU-to-Text Rank Quality and Text-to-IMU Rank Quality correspond to MRR for the two retrieval directions on the EgoExo4D zero-shot 100-sample candidate-ranking split. Word Match, Sequence Overlap, Content Alignment, and Semantic Similarity correspond to BLEU-1, ROUGE-L, METEOR, and BERTScore-F1 on the EgoExo4D zero-shot captioning split. Because these metrics have different natural ranges, the radar chart rescales each axis independently for visualization; the axis tick labels report the original metric values. ## Ablation Implementation Details {#sec:ablation_implementation_details} All ablations in `\autoref{tab:ablation}`{=latex} use the same downstream zero-shot recognition protocol, candidate label vocabularies, and metrics as the full AnyMo model. When an ablation changes an upstream stage, all downstream artifacts that depend on that stage are regenerated, including the tokenizer export, IMU-token pre-training rows, instruction-tuning rows, and evaluation tokens. This keeps the comparison focused on the removed component rather than mixing incompatible tokenizers or encoders. **Geometry-aware simulation.** The full model uses geometry-aware synthetic IMU generated from mesh-surface candidate placements, with local sensor frames estimated from surface normals and anatomical tangents as described in `\autoref{sec:synthetic_imu_generation_details}`{=latex}. For *w/o geometry-aware simulation*, we instead generate synthetic IMU from a joint-mounted setup commonly used in prior synthetic-IMU work: one default virtual sensor is rigidly attached to each of the 23 Xsens segment frames. This removes mesh-vertex candidate placements, local surface frames, normal/tangent-based orientation variation, surface-rotation augmentation, and real-device calibration. All later stages are rerun from this synthetic data: the ST-GCN encoder is trained with the same masked cross-view predictive InfoNCE objective, the PQ-VAE tokenizer is retrained on the resulting sparse-view latents, the motion-language pre-training and instruction-tuning data are re-exported with that tokenizer, and the final AnyMo model is evaluated with the same recognition protocol. Because the joint-mounted setup provides only one placement per body node, it removes within-node device-placement diversity; this ablation tests whether conventional joint-level synthetic IMU is sufficient without dense geometry-aware setup variation. **Masked cross-view predictive contrastive pre-training.** The full encoder objective samples two full-body graph views, samples sparse visible-node masks for each view, and trains a Transformer predictor to map each sparse-view latent sequence to the opposite full-view latent sequence with symmetric InfoNCE and stop-gradient targets. For *w/o masked cross-view pred. contrastive*, we keep the same synthetic graph-view sampling but replace this sparse-to-full predictive objective with a standard full-view contrastive objective. Concretely, the two independently sampled full graph views are encoded, projected, and contrasted directly against each other with sequence-level InfoNCE. No sparse visible-node mask and no sparse-to-full predictor are used in this encoder-pretraining ablation. The tokenizer, motion-language pre-training rows, instruction-tuning rows, and downstream evaluation tokens are then regenerated from this encoder. **Motion-language contrastive losses.** The full contrastive instruction-tuning stage combines three losses: the language-modeling loss on instruction rows, a narration-level symmetric IMU-text contrastive loss between IMU prompts and atomic-action narration prompts, and a supervised label-level contrastive loss between IMU prompts and curated activity-label prompts. For *w/o label contrastive*, we set the label-level contrastive loss weight to zero while keeping the narration-level contrastive loss and the language-modeling loss unchanged. For *w/o narration contrastive*, we set the narration-level contrastive loss weight to zero while keeping the label-level contrastive loss and the language-modeling loss unchanged. For *w/o all contrastive*, we remove both contrastive branches and train only with the language-modeling instruction-tuning objective from the IMU-token pre-trained checkpoint. **MCQ instruction tuning.** The full language-modeling branch is balanced between IMU-to-narration rows and IMU-to-activity multiple-choice rows. For *w/o MCQ instruction tuning*, we remove all MCQ instruction rows and replace them with additional narration rows, so that the total number of language-modeling instruction examples remains unchanged. The contrastive instruction-tuning losses and all other hyperparameters are unchanged for this ablation. # AnyMo Bench {#sec:anymo_bench} Beyond model training, AnyMo-180 also enables a controlled benchmark construction. We further derive **AnyMo Bench**, an in-the-wild HAR benchmark from real Nymeria IMU streams [@ma2024nymeria]. The benchmark is designed to stress two forms of generalization that are central to wearable sensing in realistic deployments: recognizing fine-grained daily activities on unseen subjects, and transferring across different IMU units mounted at the same body position. Such setup shifts are a persistent challenge for wearable HAR, where activity recognition can be affected by inter-subject motion variation, device hardware, exact body placement, orientation, and collection protocol [@haresamudram2025past; @cai2025towards; @bian2026foundation]. Nymeria enables this evaluation because it contains synchronized real IMU streams from three body positions: Head, Left Wrist, and Right Wrist with two co-located IMUs per position. In the cross-device setting, models train on the first IMU at each position and are tested on the second IMU at the same position, so the body placement is fixed while the IMU unit changes. **Evaluation settings.** We use a subject-disjoint 8:2 split with seed 42, yielding 157 training subjects and 39 test subjects from 196 subjects. AnyMo Bench contains 154,695 activity windows and covers 211.6 hours of real in-the-wild IMU data. The split contains 123,874 training windows and 30,965 test windows for the eligible Fine150 label space, with no class missing from either split. We report four settings: *Fine150 / Unseen Subject*, *Fine150 / Unseen Subject + Cross Device*, *Core50 / Unseen Subject*, and *Core50 / Unseen Subject + Cross Device*. In the Unseen Subject settings, train and test examples use the first IMU at each body position. In the Unseen Subject + Cross Device settings, training uses the first IMU, but testing uses the second co-located IMU for the held-out subjects. **Label spaces.** Fine150 is a fine-grained 150-class label space derived from AnyMo-180. Starting from the 180 curated classes, we remove labels that are unstable under the subject split, including classes with fewer than two test subjects or fewer than 10 test windows, and exclude a small number of highly context-dependent labels whose distinction depends heavily on visual or semantic context rather than IMU-observable motion. Fine150 remains long-tailed, with per-class counts ranging from 47 to 19,696 rows, a median of 411, and a mean of 1031.3. Core50 is a coarser 50-class label space (still more fine-grained than most existing IMU-based HAR datasets) built by merging Fine150 labels with IMU-pose-aware semantics: labels are combined when they share similar body-motion signatures, are low-sample, and remain interpretable as a coherent core activity. Large, motion-distinct, or already reliable classes are kept separate. After aggregation, Core50 class counts range from 72 to 19,696 rows, with a median of 1921 and a mean of 3093.9. ::: {#tab:anymo_bench_baselines} Model Acc@1 Acc@5 Macro-F1 --------------------------------------------------------------- -------------------- -------------------- -------------------- `\cellcolor`{=latex}*Fine150 / Unseen Subject* DeepConvLSTM 35.3 63.0 [17.2]{.underline} MantisV2 **38.5** **65.2** **22.8** COMODO [37.8]{.underline} [65.2]{.underline} 16.0 `\cellcolor`{=latex}*Fine150 / Unseen Subject + Cross Device* DeepConvLSTM 1.9 9.5 0.4 MantisV2 [14.4]{.underline} [39.7]{.underline} **8.6** COMODO **24.0** **50.6** [8.0]{.underline} `\cellcolor`{=latex}*Core50 / Unseen Subject* DeepConvLSTM 43.2 75.4 34.5 MantisV2 [45.8]{.underline} [76.8]{.underline} **41.3** COMODO **46.2** **78.8** [37.3]{.underline} `\cellcolor`{=latex}*Core50 / Unseen Subject + Cross Device* DeepConvLSTM 1.8 12.1 0.6 MantisV2 [16.6]{.underline} [48.7]{.underline} [18.4]{.underline} COMODO **32.6** **67.8** **23.3** : IMU-based HAR baseline results on AnyMo Bench. ::: **Baselines and metrics.** We evaluate three baselines. DeepConvLSTM is a classic IMU-based HAR method [@ordonez2016deep], and MantisV2 is one of the best recent time-series classification foundation models [@feofanov2026mantisv2]; both supervised IMU baselines are trained for 100 epochs on the AnyMo Bench training split. We also adapt COMODO, a self-supervised multimodal baseline for video-to-IMU representation learning [@chen2025comodo], using MantisV2 as the IMU backbone and TimeSformer as the video backbone, and train it for 20 epochs. We synchronize all IMU streams to a common 60 Hz temporal grid to align the two co-located Nymeria IMU units at each body position, while preserving temporal detail that supports the fine-grained daily-activity taxonomy [@straczkiewicz2021systematic]. The resulting activity windows are variable length, with durations concentrated around 5 seconds. Across the eligible Fine150/Core50 windows, 129,331 of 154,695 windows (83.60%) contain 300 IMU timesteps, with lengths ranging from 60 to 1200 timesteps (1.0--20.0 s). For fixed-length baselines such as DeepConvLSTM, windows shorter than 300 timesteps are zero-padded and longer windows are split into non-overlapping 300-timestep segments, dropping any remaining tail. For baselines with model-specific variable-length handling, such as MantisV2 and COMODO with a MantisV2 IMU backbone, IMU segments are resized to the model input length by interpolation. We report Acc@1, Acc@5, and macro-F1 to measure exact recognition, near-miss retrieval among many candidate activities, and class-balanced performance. **Results and implications.** As shown in `\autoref{tab:anymo_bench_baselines}`{=latex}, the results indicate that AnyMo Bench is challenging. On Fine150 / Unseen Subject, MantisV2 and COMODO reach comparable top-line accuracy, with MantisV2 at 38.5% Acc@1 and 65.2% Acc@5 and COMODO at 37.8% Acc@1 and 65.2% Acc@5, while DeepConvLSTM reaches 35.3% Acc@1 and 63.0% Acc@5. These results are meaningful for a 150-class in-the-wild unseen-subject task, but the macro-F1 scores indicate substantial remaining difficulty on tail and fine-grained classes. Core50 improves recognition accuracy, with COMODO reaching 46.2% Acc@1 and 78.8% Acc@5, while MantisV2 obtains the strongest macro-F1 at 41.3%; however, the remaining error shows that reducing the label granularity does not make the task trivial. Cross-device evaluation remains much harder: COMODO gives the strongest cross-device Acc@1, reaching 24.0% on Fine150 and 32.6% on Core50, but the gap from same-device unseen-subject recognition remains large, revealing substantial room for same-position different-device transfer. These results suggest that the difficulty comes not only from label granularity, but also from realistic subject and device shifts. Because the benchmark construction combines automatic label proposal, embedding-based semantic label consolidation, human-expert review, enum-label assignment, full class-wise auditing, relabel/drop decisions, and IMU-pose-aware label-space aggregation, AnyMo Bench provides a challenging, in-the-wild, and carefully curated testbed for future work on robust wearable motion recognition. # Downstream Evaluation Dataset Details {#sec:downstream_evaluation_dataset} The zero-shot recognition benchmark contains 14 downstream datasets that are never used for training AnyMo. `\autoref{tab:downstream_dataset_details}`{=latex} lists each dataset's activity classes and sensor placements. Sensor placements are reported using readable AnyMo graph segment names exposed to the model during evaluation, after mapping each dataset sensor to the closest node(s) in AnyMo's 23-node body graph. For EgoExo4D, Ego4D, and MMEA, we follow the train/test splits from COMODO [@chen2025comodo][^3]. For OpenPack, we use the OpenPack Challenge 2022 split [@yoshimura2024openpack][^4] and discard contiguous operation segments shorter than 1 s or longer than 30 s; this removes 204 of 10,512 valid labeled segments (1.94%) across the official split. For the remaining datasets, we use the train/test splits distributed with UniMTS [@zhang2024unimts][^5]. ```{=latex} \setlength{\tabcolsep}{2.5pt} ``` ```{=latex} \setlength{\LTcapwidth}{\linewidth} ``` ```{=latex} \renewcommand{\arraystretch}{1.08} ``` ```{=latex} \begin{longtable}{@{}C{0.12\textwidth}C{0.12\textwidth}L{0.48\textwidth}C{0.23\textwidth}@{}} \caption{Downstream zero-shot HAR datasets, activity classes, and sensor placements.} \label{tab:downstream_dataset_details}\\ \toprule Dataset & \#~Classes & Classes & Sensor Placements \\ \midrule \endfirsthead \toprule Dataset & \#~Classes & Classes & Sensor Placements \\ \midrule \endhead \bottomrule \endfoot Opportunity & 4 & stand, walk, sit, lie & L3, Right Upper Arm, Right Forearm, Left Upper Arm, Left Forearm \\ \midrule UCI-HAR & 6 & walk, walk upstairs, walk downstairs, sit, stand, lay & L5 \\ \midrule w-HAR & 7 & walk, sit, stand, jump, lie down, stairs up, stairs down & Right Foot \\ \midrule RealWorld & 8 & lying, jumping, standing, walking, sitting, climbing down, climbing up, running & Left Shoulder, Left Forearm, Head, Left Foot, Left Upper Leg, Left Upper Arm, L5 \\ \midrule TNDA-HAR & 8 & sitting, standing, lying down, ascending stairs, descending stairs, riding, walking, jogging & Right Forearm, Left Lower Leg, Right Hand, Left Foot, T8 \\ % \midrule EgoExo4D & 8 & Basketball, Bike Repair, Cooking, Dance, Health, Music, Rock Climbing, Soccer & Head \\ \midrule OpenPack & 10 & Picking, Relocate Item Label, Assemble Box, Insert Items, Close Box, Attach Box Label, Scan Label, Attach Shipping Label, Put on Back Table, Fill out Order & Right Forearm, Left Forearm, Right Upper Arm, Left Upper Arm \\ \midrule PAMAP2 & 12 & lying, sitting, standing, walking, running, cycling, nordic walking, ascending stairs, descending stairs, vacuum cleaning, ironing, rope jumping & Right Hand, T8, Right Foot \\ \midrule USC-HAD & 12 & walk forward, walk left, walk right, walk upstairs, walk downstairs, run forward, jump up, sit, stand, sleep, elevator up, elevator down & Right Upper Leg \\ \midrule WISDM & 18 & walking, jogging, stairs, sitting, standing, typing, brushing teeth, eating soup, eating chips, eating pasta, drinking cup, eating sandwich, kicking ball, playing ball, dribbling ball, writing, clapping, folding clothes & Right Hand \\ \midrule DSADS & 19 & sitting, standing, lying on back, lying on right side, ascending stairs, descending stairs, standing in an elevator still, moving around in an elevator, walking slowly, walking on a treadmill in flat positions, walking on a treadmill in inclined positions, running on a treadmill fast, exercising on a stepper, exercising on a cross trainer, cycling on an exercise bike in horizontal positions, cycling on an exercise bike in vertical positions, rowing, jumping, playing basketball & T8, Right Hand, Left Hand, Right Lower Leg, Left Lower Leg \\ \midrule UTD-MHAD & 27 & right arm swipe to the left, right arm swipe to the right, right hand wave, two hand front clap, right arm throw, cross arms in the chest, basketball shoot, right hand draw x, right hand draw circle (clockwise), right hand draw circle (counter clockwise), draw triangle, bowling (right hand), front boxing, baseball swing from right, tennis right hand forehand swing, arm curl (two arms), tennis serve, two hand push, right hand knock on door, right hand catch an object, right hand pick up and throw, jogging in place, walking in place, sit to stand, stand to sit, forward lunge (left foot forward), squat (two arms stretch out) & Right Hand for labels 0--20; Right Upper Leg for labels 21--26 \\ % \midrule Ego4D & 31 & baker, bike, bike mechanic, biology experiments, car / commuting / road trip, car / scooter washing, carpenter, cleaning / laundry, cooking, crafting / knitting / sewing / drawing / painting, cycling / jogging, eating, farmer, fixing something in the home, gardening, household management / caring for kids, indoor navigation (walking), construction / renovation jobs, playing board games, playing cards, playing games / video games, playing with pets, potting plants (indoor), practicing a musical instrument, reading books, scooter mechanic, walking on street, watching tv, working at desk, working out at home, working out outside & Head \\ \midrule MMEA & 32 & upstairs, downstairs, drinking, fall, reading, sweep floor, cut fruits, mop floor, writing, wipe table, wash hand, standing, play phone, type pc, eating, cooking, pick up phone, drop trash, fold clothes, walking, play card, brush teeth, wash dish, moving sth, type phone, chat, open close door, ride bike, sit stand, take drop sth, shopping, watch TV & Head \\ \end{longtable} ``` # Baseline Details {#sec:baseline_details} All baselines are evaluated with the same dataset-specific candidate activity vocabularies used by AnyMo. For ImageBind [@girdhar2023imagebind] and IMU2CLIP [@moon2023imu2clip], we use the checkpoints pre-trained on Ego4D for all non-Ego4D recognition datasets. For Ego4D evaluation, we instead pre-train each baseline on MMEA and evaluate the resulting checkpoint on Ego4D, avoiding an Ego4D-pre-trained checkpoint on the Ego4D target dataset. Both ImageBind and IMU2CLIP use the same 60 Hz temporal setting as AnyMo. For IMUGPT [@leng2023generating], we follow the released default configuration and use its 20 Hz IMU input setting. For HARGPT [@ji2024hargpt], we follow the default prompt-based setting and feed IMU at 10 Hz. For UniMTS [@zhang2024unimts], we use the released checkpoint generated and pre-trained at 20 Hz, and evaluate it under the same 20 Hz setting. For NormWear [@luo2026toward], we follow the official implementation and its 65 Hz input setting. For the Gemma 4 26B text and plot baselines [@googledeepmind2026gemma4modelcard], we downsample IMU inputs to 10 Hz; even at 10 Hz, we use a 65,536-token context window so that all benchmark datasets can be processed without truncation. # Prompt Analysis ## Prompt Sensitivity {#sec:prompt_sensitivity} Zero-shot recognition embeds each candidate activity label through the language backbone, so the text-side representation is sensitive to how the label is prompted. To quantify this effect, we evaluate four fixed prompt formats with the same recognition protocol as `\autoref{tab:zeroshot_har}`{=latex}. The fixed prompts are: *bare label*, which uses the class name directly; *person prompt*, \`\`a person is {label}"; *IMU prompt*, \`\`wearable IMU motion of {label}"; and *activity prompt*, \`\`the activity is {label}". We compare them with the final AnyMo setting, which replaces manual text templates with an 8-token learnable soft prompt on the text/label branch. ::: {#tab:prompt_sensitivity} Text-side prompt Acc F1 R@2 -------------------------- ---------- ---------- ---------- Bare label 33.1 25.9 54.3 Person prompt 35.1 28.4 54.8 IMU prompt 28.3 22.5 47.0 Activity prompt 35.2 28.0 56.1 Learnable prompt (AnyMo) **35.7** **29.5** **57.5** : Prompt sensitivity for zero-shot HAR. Results averaged over 14 downstream datasets. ::: The fixed prompts produce noticeably different rankings even though the label vocabulary, IMU embeddings, and evaluation protocol are unchanged. For example, the IMU-centric template is weaker than the person- and activity-centric templates on this benchmark, indicating that manually written label contexts can shift the text embeddings in ways that are not consistently beneficial for recognition. The learnable soft prompt avoids committing to a hand-written template: during contrastive instruction tuning, it learns a task-specific textual context for activity labels that better matches the IMU embedding space. This also removes the need to choose a dataset-specific prompt template at evaluation time. Accordingly, AnyMo uses the learnable prompt in all final zero-shot recognition results, where it gives the best average Acc, F1, and Recall@2 among the compared single-prompt settings. ## Prompt Templates {#sec:prompt_templates} `\autoref{fig:prompt_templates}`{=latex} summarizes the text templates used by AnyMo. The IMU-token language-model pre-training stage is trained to generate IMU token sequences. The narration and MCQ instruction templates are used during language-modeling instruction tuning; captioning evaluation uses the narration template, and MCQ-style evaluation uses the MCQ template. The contrastive IMU and text templates are used during contrastive instruction tuning and are used for retrieval and zero-shot recognition, with the learnable soft prompt prepended to the text/label branch in the final AnyMo checkpoint. `\newcommand{\prompttemplatebox}[2]{% \noindent\fbox{% \colorbox{black!75}{\parbox{\dimexpr\linewidth-2\fboxsep\relax}{\strut\color{white}\bfseries #1}} \par #2 \par }}`{=latex} ```{=latex} \begin{figure*}[h] \small \prompttemplatebox{IMU-Token LM Pre-Training}{ \texttt{\{IMU\_TOKEN\_SEQUENCE\}} } \prompttemplatebox{Narration Instruction Tuning / Captioning Evaluation}{ Describe the human motion represented by the wearable IMU motion tokens.\newline The IMU tokens are from IMU sensors attached to the user's \texttt{\{SENSOR\_CONTEXT\}}.\newline Input IMU token:\newline \texttt{\{IMU\_TOKEN\_SEQUENCE\}}\newline \textbf{Assistant target:} \texttt{\{MOTION\_NARRATION\}} } \prompttemplatebox{MCQ Instruction Tuning / MCQ Evaluation}{ Recognize the activity represented by the wearable IMU motion tokens.\newline The IMU tokens are from IMU sensors attached to the user's \texttt{\{SENSOR\_CONTEXT\}}.\newline Input IMU token:\newline \texttt{\{IMU\_TOKEN\_SEQUENCE\}}\newline CHOICES:\newline \texttt{\{CHOICE\_KEY\}: \{ACTIVITY\_LABEL\}}\newline Choose the best matching option. Output the option key followed by the selected activity label.\newline \textbf{Assistant target:} \texttt{\{ANSWER\_KEY\}: \{ACTIVITY\_LABEL\}} } \prompttemplatebox{Contrastive IMU Prompt}{ Represent the human motion from the wearable IMU motion tokens.\newline The IMU tokens are from IMU sensors attached to the user's \texttt{\{SENSOR\_CONTEXT\}}.\newline Input IMU token:\newline \texttt{\{IMU\_TOKEN\_SEQUENCE\}}\newline Return a compact embedding of the motion. } \prompttemplatebox{Contrastive Text / Label Prompt}{ Represent the human motion described by the text.\newline Motion description:\newline \texttt{\{NARRATION\_OR\_ACTIVITY\_LABEL\}}\newline Return a compact embedding of the motion. } \caption{Prompt templates used for motion-language pre-training, instruction tuning, contrastive tuning, and evaluation.} \label{fig:prompt_templates} \end{figure*} ``` [^1]: Corresponding authors: breeze.chen@unsw.edu.au, flora.salim@unsw.edu.au. [^2]: [^3]: [^4]: [^5]: