VENOM: Versatile Embodied Network for Omni-bodied Motion tracking

2026-06-15

AI Summary

This work addresses the challenge of achieving expert-level whole-body motion tracking across multiple humanoid robots using only demonstration data. The authors propose an end-to-end cross-embodiment motion tracking model based on the GPT architecture, establishing the first unified framework that does not require decoupling upper and lower body control. They introduce the VENOM dataset, which encompasses multi-robot states, actions, and rewards, enabling the model to learn policies directly from heterogeneous demonstrations through sequence modeling and supervised learning—bypassing reliance on reinforcement learning reward signals. Experimental results demonstrate that the proposed method achieves stable tracking performance across diverse humanoid platforms, significantly outperforming MLP baselines and approaching the performance of expert policies trained via asymmetric Actor-Critic reinforcement learning.

Abstract

Achieving expert-level expressive full-body motion tracking across multiple humanoids solely from demonstration data remains a challenging and relatively an underexplored problem in humanoid robot learning. Cross-embodiment motion tracking policies are mostly trained by decoupling the control problem into upper and lower body control. This work proposes VENOM, a cross-embodiment full-body motion tracking model for humanoids in simulation. VENOM is a GPT-based motion tracker trained on multiple humanoid data that can track the entire body without the requirement to split into upper and lower body control. We curate a multi-humanoid motion tracking dataset called the VENOM dataset that contains states, actions, and rewards and train VENOM and the baselines on this dataset. In this letter, we evaluate VENOM's performance against baselines and show that we can achieve a stable motion tracker across different humanoids more capable than an MLP trained on multiple humanoid data with supervised learning alone, and also show that despite lack of reward feedback, VENOM closely matches the tracking capability of experts that were trained using asymmetric-actor critic reinforcement learning.

Problem

• motion tracking
• humanoid robots
• cross-embodiment
• full-body control
• demonstration data

Innovation

• cross-embodiment
• full-body motion tracking
• GPT-based policy
• humanoid robots
• supervised learning