Odoriko: A Shape-Aware Multimodal Diffusion Framework for Human Motion

2026-06-19

AI Summary

Existing unified multimodal human motion generation methods overlook individual biomechanical differences—such as gender and body shape—making it difficult to produce personalized motions. To address this limitation, this work proposes Odoriko, the first unified multimodal diffusion framework that explicitly incorporates morphological information to generate motions consistent with an individual’s body shape across diverse modalities, including text, music, and video. Notably, when morphological data is unavailable, Odoriko jointly estimates body morphology and synthesizes corresponding motions in a synchronized manner. Experimental results demonstrate that Odoriko matches or surpasses specialized models in tasks such as text-to-motion, music-to-dance, and video-to-motion generation, while uniquely enabling morphology-consistent motion synthesis—a capability absent in current approaches.

Abstract

Human motion generation has been widely studied across diverse input modalities, text, music, and video, and recent efforts have unified these into single multimodal frameworks. However, while morphological factors such as gender and body shape are known to produce distinct kinematic signatures, no existing unified framework incorporates this into generation, treating all subjects as morphologically equivalent. We present Odoriko, the first unified multimodal motion generation framework that reflects subject bio-morphological information directly in synthesized motion output. Rather than averaging over subject variation, Odoriko generates motion that is consistent with who is moving, not just what they are asked to do, across text, music, and video conditions within a single model. When explicit morphological information is unavailable, Odoriko additionally recovers subject morphology alongside motion, unifying estimation and generation in one framework. Extensive experiments across text-to-motion, music-to-dance, and video-to-motion benchmarks demonstrate that Odoriko matches or exceeds prior specialized models on standard metrics, while enabling morphology-consistent generation that no existing unified framework supports.

Problem

• human motion generation
• multimodal framework
• body shape
• morphological variation
• motion synthesis

Innovation

• shape-aware
• multimodal diffusion
• human motion generation
• morphology-consistent
• unified framework