Bridging the Embodiment Gap: Disentangled Cross-Embodiment Video Editing

2026-05-05

AI Summary

This work addresses the distribution shift arising from morphological differences between humans and robots by proposing a disentangled cross-embodiment video generation framework. The approach decomposes human demonstration videos into two orthogonal latent spaces representing task semantics and embodiment-specific motion. Through dual contrastive learning, mutual information minimization, and orthogonality constraints, the method explicitly disentangles these representations. By integrating parameter-efficient adapters into a frozen video diffusion model, it enables the generation of temporally coherent and morphologically accurate robot execution videos from a single human demonstration. To the best of our knowledge, this is the first method to achieve task–embodiment disentanglement without requiring paired cross-embodiment data, substantially enhancing the scalability and adaptability of robot learning from large-scale in-the-wild human videos.

Abstract

Learning robotic manipulation from human videos is a promising solution to the data bottleneck in robotics, but the distribution shift between humans and robots remains a critical challenge. Existing approaches often produce entangled representations, where task-relevant information is coupled with human-specific kinematics, limiting their adaptability. We propose a generative framework for cross-embodiment video editing that directly addresses this by learning explicitly disentangled task and embodiment representations. Our method factorizes a demonstration video into two orthogonal latent spaces by enforcing a dual contrastive objective: it minimizes mutual information between the spaces to ensure independence while maximizing intra-space consistency to create stable representations. A parameter-efficient adapter injects these latent codes into a frozen video diffusion model, enabling the synthesis of a coherent robot execution video from a single human demonstration, without requiring paired cross-embodiment data. Experiments show our approach generates temporally consistent and morphologically accurate robot demonstrations, offering a scalable solution to leverage internet-scale human video for robot learning.

Problem

• embodiment gap
• cross-embodiment
• disentangled representation
• robotic manipulation
• video editing

Innovation

• disentangled representation
• cross-embodiment
• video diffusion model
• contrastive learning
• robotic manipulation