Progressively Learning Heterogeneous Skills in a Unified Latent Space

2026-08-24

AI Summary

提出HetSkills框架，通过统一潜在空间逐步学习异构技能，解决物理角色控制中技能整合问题，实验显示其在多种任务中有效。

Abstract

We propose HetSkills, a novel framework designed to progressively learn heterogeneous skills within a unified latent space for physics-based character control. The core idea is to treat this latent space as a shared executable interface, enabling seamless integration of skills learned from diverse data sources, supervision forms, and tasks. HetSkills begins by learning a tracking skill that establishes a strong foundation in motion control and creates a shared motion decoder, which can be reused across tasks without the need for retraining or separate controllers. To prevent the text-to-motion skill from exploiting shortcut pathways instead of learning language semantics, we introduce motion intuition distillation to ground text-to-motion generation in language semantics and a task-guidance module that dynamically adjusts actions based on high-level language instructions. This enables HetSkills to preserve natural motion while continuously expanding its skill repertoire, making it highly adaptable for long-horizon tasks. Experimental results demonstrate the effectiveness in motion tracking, text-to-motion generation, motion completion, and downstream task adaptation, achieving impressive success rates even under challenging conditions.

Problem

• heterogeneous skills
• unified latent space
• physics-based character control
• text-to-motion generation
• motion tracking

Innovation

• HetSkills
• unified latent space
• motion intuition distillation
• task-guidance module
• heterogeneous skills