AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild

2026-05-21

AI Summary

This work addresses the challenge of generalizing motion signals collected from wearable devices due to heterogeneous sensor placements. To overcome this, the authors propose AnyMo, a framework that leverages physics-driven IMU simulation to generate diverse synthetic data for pretraining a graph encoder. It unifies multi-position IMU signals into full-body motion tokens aligned with a large language model, enabling joint motion-language understanding. AnyMo is the first method to support unified modeling of IMU signals under arbitrary deployment configurations, achieving zero-shot activity recognition across 14 unseen datasets with an average accuracy gain of 11.7%, cross-modal IMU-to-text retrieval with up to 28.6% higher MRR, and motion captioning with an 18.8% improvement in BERT-F1 score.

Abstract

As wearable and mobile devices become increasingly embedded in daily life, they offer a practical way to continuously sense human motion in the wild. But inertial signals are highly dependent on the sensing setup, including body location, mounting position, sensor orientation, device hardware, and sampling protocol. This setup dependence makes it difficult to learn motion representations that transfer across devices and datasets, and limits the broader use of wearable IMUs beyond closed-set recognition. We introduce AnyMo, a geometry-aware framework for setup-agnostic human motion modeling. AnyMo uses physics-grounded IMU simulation over dense body-surface placements to generate diverse and plausible synthetic signals, pre-trains a graph encoder from paired synthetic placement views and masked partial observations, tokenizes multi-position IMU into full-body motion tokens, and aligns these tokens with an LLM for motion-language understanding. We evaluate AnyMo on three complementary tasks: zero-shot activity recognition across 14 unseen downstream datasets, cross-modal retrieval, and wearable IMU motion captioning, where it improves average Accuracy/F1/R@2 by 11.7\%/11.6\%/22.6\% on HAR, increases zero-shot IMU-to-text and text-to-IMU retrieval MRR by 15.9\% and 28.6\%, respectively, and improves zero-shot captioning BERT-F1 by 18.8\%. These results support AnyMo as a generalist model for wearable motion understanding in the wild. Project page: https://baiyuchen.com/project/AnyMo.

Problem

• inertial signals
• setup dependence
• motion representation
• wearable IMUs
• cross-dataset transfer

Innovation

• setup-agnostic
• geometry-aware
• IMU simulation
• motion-language alignment
• graph encoder