Human Motion Capture from Loose and Sparse Inertial Sensors with Garment-aware Diffusion Models

2025-06-18

AI Summary

Accurate full-body pose estimation from sparsely and loosely worn inertial measurement units (IMUs) remains challenging due to severe signal distortion induced by clothing motion and sensor slippage—contravening the conventional assumption of skin-tight sensor placement. Method: This work formally introduces the novel task of “loose-IMU motion capture” and proposes a clothing-aware physical simulation framework to model garment deformation effects on IMU measurements. We design a Transformer-based diffusion model that jointly learns loose-signal generation and pose inversion, trained exclusively on synthetically generated data with realistic clothing dynamics. Contribution/Results: Our method achieves high-fidelity full-body pose estimation using only 4–6 loosely mounted IMUs. Extensive experiments demonstrate consistent and significant improvements over state-of-the-art approaches across multiple benchmarks, validating the feasibility of lightweight, robust inertial motion capture under realistic, unconstrained wearing conditions.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Web data generation and simulation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Motion capture using sparse inertial sensors has shown great promise due to its portability and lack of occlusion issues compared to camera-based tracking. Existing approaches typically assume that IMU sensors are tightly attached to the human body. However, this assumption often does not hold in real-world scenarios. In this paper, we present a new task of full-body human pose estimation using sparse, loosely attached IMU sensors. To solve this task, we simulate IMU recordings from an existing garment-aware human motion dataset. We developed transformer-based diffusion models to synthesize loose IMU data and estimate human poses based on this challenging loose IMU data. In addition, we show that incorporating garment-related parameters while training the model on simulated loose data effectively maintains expressiveness and enhances the ability to capture variations introduced by looser or tighter garments. Experiments show that our proposed diffusion methods trained on simulated and synthetic data outperformed the state-of-the-art methods quantitatively and qualitatively, opening up a promising direction for future research.

Problem

• Estimating human pose from loosely attached IMU sensors
• Simulating loose IMU data for motion capture
• Incorporating garment parameters to improve pose estimation

Innovation

• Garment-aware diffusion models for motion capture
• Transformer-based synthesis of loose IMU data
• Incorporating garment parameters to enhance accuracy