LeVR: A Modular VR Teleoperation Framework for Imitation Learning in Dexterous Manipulation

2025-09-17

AI Summary

Current robotic imitation learning faces two key bottlenecks: (1) a lack of robust VR-based teleoperation solutions tailored for dexterous hands, and (2) insufficient native integration with state-of-the-art imitation learning frameworks such as LeRobot. To address these, we propose the first modular VR teleoperation framework natively integrated with LeRobot, enabling synchronized control of the Franka arm and RobotEra XHand dexterous manipulator. Our framework supports intuitive, high-fidelity demonstration capture and end-to-end policy deployment, closing the loop from VR data collection to model training and real-world execution. We release an open-source dataset comprising 100 expert demonstrations, significantly improving visual-motor policy fine-tuning efficiency. Key contributions include: (1) the first native LeRobot-compatible VR interface; (2) a low-latency, high-precision teleoperation architecture designed specifically for dexterous manipulation; and (3) a reproducible, extensible open-source paradigm unifying hardware and software co-design.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Computer Vision: Vision for Robotics & Autonomous Driving

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

We introduce LeVR, a modular software framework designed to bridge two critical gaps in robotic imitation learning. First, it provides robust and intuitive virtual reality (VR) teleoperation for data collection using robot arms paired with dexterous hands, addressing a common limitation in existing systems. Second, it natively integrates with the powerful LeRobot imitation learning (IL) framework, enabling the use of VR-based teleoperation data and streamlining the demonstration collection process. To demonstrate LeVR, we release LeFranX, an open-source implementation for the Franka FER arm and RobotEra XHand, two widely used research platforms. LeFranX delivers a seamless, end-to-end workflow from data collection to real-world policy deployment. We validate our system by collecting a public dataset of 100 expert demonstrations and use it to successfully fine-tune state-of-the-art visuomotor policies. We provide our open-source framework, implementation, and dataset to accelerate IL research for the robotics community.

Problem

• Enables VR teleoperation for dexterous robotic manipulation data collection
• Integrates VR data with imitation learning framework for streamlined training
• Provides open-source implementation for real-world policy deployment workflow

Innovation

• Modular VR teleoperation for dexterous manipulation
• Native integration with LeRobot imitation learning framework
• Open-source implementation for Franka arm and XHand