Improving Sparse IMU-based Motion Capture with Motion Label Smoothing

2025-11-27

AI Summary

To address the underexplored problem of regularization in sparse-IMU human motion capture, this paper introduces label smoothing—the first such application to this task—and proposes a skeletal-structure-aware Perlin noise mixing strategy. The method deliberately increases label entropy while preserving motion temporal smoothness, joint topological consistency, and low-frequency dominance. It is architecture-agnostic, seamlessly integrating with both RNN- and Transformer-based models. Extensive experiments across four real-world IMU datasets demonstrate consistent improvements in motion reconstruction accuracy—averaging 2.1–4.7 mm reduction in joint position error—across three state-of-the-art models, alongside enhanced robustness to sensor noise and motion variability. Notably, the approach exhibits strong cross-model generalization, effectively bridging a critical gap in motion-prior-driven regularization for IMU-based motion capture.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Sparse Inertial Measurement Units (IMUs) based human motion capture has gained significant momentum, driven by the adaptation of fundamental AI tools such as recurrent neural networks (RNNs) and transformers that are tailored for temporal and spatial modeling. Despite these achievements, current research predominantly focuses on pipeline and architectural designs, with comparatively little attention given to regularization methods, highlighting a critical gap in developing a comprehensive AI toolkit for this task. To bridge this gap, we propose motion label smoothing, a novel method that adapts the classic label smoothing strategy from classification to the sparse IMU-based motion capture task. Specifically, we first demonstrate that a naive adaptation of label smoothing, including simply blending a uniform vector or a ``uniform'' motion representation (e.g., dataset-average motion or a canonical T-pose), is suboptimal; and argue that a proper adaptation requires increasing the entropy of the smoothed labels. Second, we conduct a thorough analysis of human motion labels, identifying three critical properties: 1) Temporal Smoothness, 2) Joint Correlation, and 3) Low-Frequency Dominance, and show that conventional approaches to entropy enhancement (e.g., blending Gaussian noise) are ineffective as they disrupt these properties. Finally, we propose the blend of a novel skeleton-based Perlin noise for motion label smoothing, designed to raise label entropy while satisfying motion properties. Extensive experiments applying our motion label smoothing to three state-of-the-art methods across four real-world IMU datasets demonstrate its effectiveness and robust generalization (plug-and-play) capability.

Problem

• Addresses lack of regularization in sparse IMU motion capture
• Proposes motion label smoothing to enhance label entropy
• Introduces skeleton-based Perlin noise preserving motion properties

Innovation

• Motion label smoothing adapts classification strategy to motion capture
• Uses skeleton-based Perlin noise to increase label entropy
• Preserves temporal smoothness, joint correlation, and low-frequency dominance