Beyond MoCap: Scaling Motion Tokenizers with Synthetic Human Motion for Generative Modeling

2026-06-25

AI Summary

Existing motion capture datasets suffer from limited diversity, which constrains the generalization capabilities of generative models on rare, highly dynamic, and compositionally complex actions. To address this limitation, this work proposes a method that leverages large-scale synthetic human motion data combined with physics-based plausibility constraints to jointly expand both the training distribution and the size of the discrete codebook. By reconstructing the VQ-VAE motion tokenizer beyond the confines of real-data distributions, the approach substantially broadens the coverage and compositional capacity of the discrete motion representation space. This leads to consistent performance gains in text-to-motion generation and motion in-betweening tasks, and the enhanced representations can be seamlessly integrated into existing frameworks such as MotionGPT, demonstrating both the effectiveness and generalizability of the proposed representation expansion.

Abstract

Human motion generation models are fundamentally constrained by the limited diversity of motion capture datasets, which predominantly contain common, repetitive actions and fail to cover the long tail of complex human movements, resulting in a restricted motion vocabulary in learned latent representations and poor generalization to rare, compositional, and highly dynamic motions. In this work, we propose a framework for expanding the motion representation space by leveraging large-scale synthetic human motion, introducing a data generation pipeline that produces diverse, physically plausible motion sequences beyond the distribution of existing datasets and integrating it with a redesigned VQ-VAE tokenizer that adapts to this expanded motion space. Unlike conventional tokenizers trained on narrow data distributions, our approach jointly scales both the training distribution and the discrete codebook, enabling the model to capture a significantly richer set of motion primitives. We demonstrate that training with synthetic motion substantially improves the coverage and compositionality of the learned motion vocabulary, leading to consistent gains across motion generation tasks such as text-to-motion and motion continuation, while remaining fully compatible with existing frameworks including MotionGPT. Our results suggest that the primary bottleneck lies in the limited support of the learned motion representation, rather than model architecture alone. Scaling synthetic motion in tandem with representation learning offers a principled path toward more expressive, controllable, and generalizable human motion synthesis.

Problem

• motion capture
• motion generation
• synthetic human motion
• motion representation
• long-tail motions

Innovation

• synthetic human motion
• motion tokenization
• VQ-VAE
• motion generation
• representation scaling