HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generation

2025-12-29

AI Summary

This work addresses three key challenges in text-to-3D human motion generation: low motion fidelity, weak instruction alignment, and narrow action category coverage. We propose the first billion-parameter DiT-based flow matching model and introduce a novel three-stage training paradigm—pretraining, fine-tuning, and Reinforcement Learning from Human Feedback (RLHF). Pretraining leverages large-scale, multi-source cleaned motion data with precise caption annotations; RLHF employs a learnable reward model jointly optimized with human feedback to significantly enhance instruction following. Our framework achieves the broadest open-source coverage to date—spanning six high-level action categories and over 200 fine-grained motion classes—and outperforms all prior state-of-the-art methods on major benchmarks. To foster reproducibility and industrial adoption, we fully open-source our code, models, and datasets.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Generation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We present HY-Motion 1.0, a series of state-of-the-art, large-scale, motion generation models capable of generating 3D human motions from textual descriptions. HY-Motion 1.0 represents the first successful attempt to scale up Diffusion Transformer (DiT)-based flow matching models to the billion-parameter scale within the motion generation domain, delivering instruction-following capabilities that significantly outperform current open-source benchmarks. Uniquely, we introduce a comprehensive, full-stage training paradigm -- including large-scale pretraining on over 3,000 hours of motion data, high-quality fine-tuning on 400 hours of curated data, and reinforcement learning from both human feedback and reward models -- to ensure precise alignment with the text instruction and high motion quality. This framework is supported by our meticulous data processing pipeline, which performs rigorous motion cleaning and captioning. Consequently, our model achieves the most extensive coverage, spanning over 200 motion categories across 6 major classes. We release HY-Motion 1.0 to the open-source community to foster future research and accelerate the transition of 3D human motion generation models towards commercial maturity.

Problem

• Generates 3D human motions from textual descriptions
• Scales flow matching models to billion parameters for motion
• Ensures precise text-motion alignment via full-stage training

Innovation

• Scaling billion-parameter Diffusion Transformer flow matching models
• Full-stage training with pretraining, fine-tuning, and reinforcement learning
• Meticulous data processing for extensive motion category coverage