MuGen: Multi-Skill Generative Locomotion Controller for Humanoid Robots

2026-05-23

AI Summary

This work proposes MuGen, a framework designed to enable humanoid robots to achieve human-like motion control across a diverse repertoire of skills. By integrating a vector-quantized variational autoencoder (VQ-VAE) with model-based reinforcement learning, MuGen constructs a structured latent motion space that captures essential movement representations from human demonstration data. A teacher-student policy distillation mechanism efficiently transfers these data-driven generative motion representations into deployable control policies. The framework allows robots to faithfully reproduce complex motion sequences with high fidelity and demonstrates strong generalization to unseen human motions, significantly enhancing both the expressiveness and scalability of humanoid locomotion and manipulation behaviors.

Abstract

This paper presents MuGen, a data-driven framework for learning and deploying multi-skill locomotion on humanoid robots. MuGen enables a robot to perform expressive motions like humans under the guidance of example motion sequences. To achieve this, we employ vector-quantized autoencoders (VQ-VAEs) trained with model-based reinforcement learning, resulting in a generative representation of locomotion that captures key patterns of human motion from hours of heterogeneous human performance data. We employ a teacher-student learning framework and develop a new policy distillation strategy to enable a deployable student policy learning this efficient latent representation. This policy allows the robot to track and mimic unseen human motions and further enables the robot to reuse the learned latent space for other tasks. We demonstrate the effectiveness of our framework through a diverse set of motions and accurate execution.

Problem

• humanoid robots
• multi-skill locomotion
• expressive motion
• motion imitation
• generative control

Innovation

• VQ-VAE
• model-based reinforcement learning
• policy distillation
• latent motion representation
• multi-skill locomotion