RoboCOIN: An Open-Sourced Bimanual Robotic Data COllection for INtegrated Manipulation

2025-11-21

AI Summary

To address the scarcity of high-quality, large-scale manipulation data for bimanual robotic systems—stemming from hardware heterogeneity and hindering dexterous manipulation research—this paper introduces the largest open-source bimanual robotic manipulation dataset to date. It encompasses 15 heterogeneous robot platforms, 16 operational scenarios, and 421 distinct tasks, with over 180,000 expert demonstrations. We propose a hierarchical capability pyramid annotation framework enabling multi-granularity semantic labeling at trajectory-, segment-, and frame-levels. Additionally, we design the Robot Trajectory Markup Language (RTML) and the unified management framework CoRobot, which support standardized parsing, automated annotation generation, and quality assessment of heterogeneous data. Experimental results demonstrate that our dataset significantly improves performance across diverse models on cross-platform bimanual manipulation tasks, exhibiting strong generalization and reliability.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Safety and Robustness

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

Bimanual manipulation is essential for achieving human-like dexterity in robots, but the large-scale and diverse bimanual robot datasets remain scarce due to hardware heterogeneity across robotic platforms. To address the challenge, we present RoboCOIN, a comprehensive multi-embodiment bimanual manipulation dataset with over 180,000 demonstrations collected from 15 distinct robotic platforms. The dataset covers 16 scenarios, including residential, commercial, and working environments, with 421 tasks systematically organized by bimanual coordination patterns and object properties. Our key innovation is a hierarchical capability pyramid that provides multi-level annotations, spanning trajectory-level concepts, segment-level subtasks, and frame-level kinematics. We further develop CoRobot, a comprehensive processing framework featuring Robot Trajectory Markup Language (RTML) for quality assessment, automated annotation generation, and unified multi-embodiment management. Extensive experiments demonstrate the reliability and effectiveness of RoboCOIN in multi-embodiment bimanual learning, with significant performance improvements across various model architectures and robotic platforms. The complete dataset and framework are open-sourced and publicly available for further research purposes. Project website: https://FlagOpen.github.io/RoboCOIN/.

Problem

• Addresses scarcity of large-scale bimanual robot datasets
• Presents multi-embodiment dataset with 180,000+ robotic demonstrations
• Develops hierarchical annotation framework for manipulation learning

Innovation

• Hierarchical capability pyramid for multi-level annotations
• RTML framework for quality assessment and management
• Multi-embodiment dataset with 180,000 bimanual demonstrations