TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System

2025-11-04

AI Summary

Humanoid robotics has long been hindered by costly and inefficient data collection frameworks; existing teleoperation systems typically rely on motion-capture hardware or decoupled control, limiting their ability to capture full-body coordinated skills. This paper introduces the first lightweight, scalable, markerless whole-body haptic teleoperation system: it leverages the PICO 4 Ultra headset for real-time full-body pose estimation, integrates a custom-built 2-DoF neck actuator enabling active visual perception and first-person viewpoint alignment, and employs a hierarchical vision–motor policy model for closed-loop autonomous control. The system successfully demonstrates dexterous manipulation and dynamic kicking—complex full-body tasks—with ≈100% success rate over 100 high-quality demonstrations collected within 15 minutes. To foster community advancement, we fully open-source the code, hardware designs, and dataset—establishing a new paradigm for scalable skill acquisition in humanoid robotics.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Motion & Tracking

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Large-scale data has driven breakthroughs in robotics, from language models to vision-language-action models in bimanual manipulation. However, humanoid robotics lacks equally effective data collection frameworks. Existing humanoid teleoperation systems either use decoupled control or depend on expensive motion capture setups. We introduce TWIST2, a portable, mocap-free humanoid teleoperation and data collection system that preserves full whole-body control while advancing scalability. Our system leverages PICO4U VR for obtaining real-time whole-body human motions, with a custom 2-DoF robot neck (cost around $250) for egocentric vision, enabling holistic human-to-humanoid control. We demonstrate long-horizon dexterous and mobile humanoid skills and we can collect 100 demonstrations in 15 minutes with an almost 100% success rate. Building on this pipeline, we propose a hierarchical visuomotor policy framework that autonomously controls the full humanoid body based on egocentric vision. Our visuomotor policy successfully demonstrates whole-body dexterous manipulation and dynamic kicking tasks. The entire system is fully reproducible and open-sourced at https://yanjieze.com/TWIST2 . Our collected dataset is also open-sourced at https://twist-data.github.io .

Problem

• Humanoid robotics lacks scalable data collection frameworks for whole-body control
• Existing teleoperation systems use decoupled control or expensive motion capture setups
• Need portable mocap-free system for holistic human-to-humanoid teleoperation and data collection

Innovation

• Mocap-free VR teleoperation for whole-body control
• Custom 2-DoF robot neck for egocentric vision
• Hierarchical visuomotor policy for autonomous humanoid control