MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations

2026-06-22

AI Summary

This work addresses the challenges of structural distortion and fine-detail loss in human motion sequence reconstruction from sparse observations by proposing a coarse-to-fine, multi-scale autoregressive framework. The method first learns hierarchical motion representations using a Temporal Multi-scale Tokenization VQ-VAE, then employs a Scale-Aware Control module to guide global trajectory generation with sparse input cues, and finally refines high-frequency motion details through a Motion Autoregressive Network in a progressive manner. Experimental results on the AMASS dataset demonstrate that the proposed approach significantly outperforms existing methods, accurately recovering intricate motion patterns while preserving semantic and structural consistency. This study establishes a new paradigm for structure-aware human motion reconstruction under sparse observation conditions.

Abstract

Human motion follows a temporal hierarchical structure, transitioning from low-frequency global trajectories to high-frequency details. Inspired by the success of multi-level autoregressive models in computer vision, we propose MotionMAR, a coarse-to-fine framework for motion reconstruction from sparse observations. It first estimates the global trajectory of human motion and then gradually refines the temporal details. This architecture consists of four integrated components. The Temporal Multi-scale Tokenization (TMT) VQ-VAE encodes the data at multiple temporal resolutions, separating semantic motion from minor jitters. The Motion Autoregressive Network (MAN) operates in this latent space, predicting motion across scales. It first establishes the global structure through coarse indices and then generates finer indices to recover specific details. Meanwhile, the Scale-Aware Control (SAC) module integrates sparse tracking data to ensure the generated output aligns with actual observations. The Motion Refinement Network (MRN) subsequently smooths consecutive poses and eliminates quantization artifacts. Experiments show that MotionMAR achieves state-of-the-art accuracy on the AMASS dataset, providing a reliable and structure-aware approach for motion reconstruction. The source code is publicly available at http://www.lidarhumanmotion.net/motionmar/.

Problem

• human motion reconstruction
• sparse observations
• temporal hierarchy
• motion details
• global trajectory

Innovation

• multi-scale autoregressive
• motion reconstruction
• temporal hierarchical modeling
• VQ-VAE
• sparse observations