DC-Motion: Decoupling Semantics and Details via Discrete-Continuous Tokens for Human Motion Generation

2026-05-28

AI Summary

Existing text-to-motion generation methods struggle to simultaneously model global structural coherence and fine-grained motion details, often compromising between controllability and fidelity. This work proposes a discrete-continuous decoupled framework that explicitly separates the compositional semantics of actions from the continuous dynamics of motion trajectories: discrete tokens represent the global action structure and are generated via masked modeling, while continuous residual latent variables capture local dynamics and are modeled using a diffusion process. The resulting two-stage conditional generation architecture significantly outperforms state-of-the-art baselines on the HumanML3D and KIT-ML datasets, achieving the best performance in terms of both FID and R-Precision metrics.

Abstract

Text-to-motion generation requires synthesizing physically realistic dynamics that strictly follow complex and long-horizon textual instructions. Existing approaches rely on homogeneous representation spaces that may fail to capture the hierarchical nature of human motion, with diffusion models struggling at compositional semantic reasoning and AR models sacrificing fine-grained physical details due to quantization. To solve it, we introduce DC-Motion, a factorized generative framework designed to explicitly decouple semantics and details via discrete-continuous tokens. A Discrete-Continuous VAE (DC-VAE) first decomposes motion into discrete tokens for semantics and continuous residuals for fine-grained dynamics. Then, a masked AR model predicts the discrete structure from text, and a lightweight residual diffusion model recovers the continuous physical details. Extensive experiments demonstrate that DC-Motion effectively improves the capability to follow complex instructions. By effectively balancing semantic controllability and physical realism, our approach offers a highly adaptable modeling paradigm for human motion generation. On both HumanML3D and KIT-ML datasets, DC-Motion achieves state-of-the-art performance, delivering the best FID for motion realism and R-precision for text alignment.

Problem

• text-to-motion generation
• discrete-continuous representation
• action structure
• motion dynamics
• human motion generation

Innovation

• discrete-continuous representation
• motion generation
• structure-detail decoupling
• text-to-motion
• diffusion model