Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics

2026-04-22

AI Summary

This work addresses the longstanding scarcity of large-scale, multi-platform, open-access high-quality datasets in medical robotics, which has hindered the development of foundational models. To bridge this gap, we present the largest open dataset to date of medical robot videos paired with synchronized kinematic data, encompassing 49 institutions and diverse surgical robot platforms, with a novel unified cross-platform integration framework. Leveraging this dataset, we introduce GR00T-H, the first open-source vision–language–action foundation model for medical robotics, achieving a 25% end-to-end success rate (versus 0% for baselines) on structured suturing tasks and completing on average 64% of a 29-step ex vivo suturing sequence. We also propose Cosmos-H, a unified world model supporting simulation and policy evaluation across nine robotic platforms.

Technology Category

• Intelligent Robots: Other Foundations and Applications
• Computer Vision: Vision for Robotics & Autonomous Driving
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Autonomous medical robots hold promise to improve patient outcomes, reduce provider workload, democratize access to care, and enable superhuman precision. However, autonomous medical robotics has been limited by a fundamental data problem: existing medical robotic datasets are small, single-embodiment, and rarely shared openly, restricting the development of foundation models that the field needs to advance. We introduce Open-H-Embodiment, the largest open dataset of medical robotic video with synchronized kinematics to date, spanning more than 49 institutions and multiple robotic platforms including the CMR Versius, Intuitive Surgical's da Vinci, da Vinci Research Kit (dVRK), Rob Surgical BiTrack, Virtual Incision's MIRA, Moon Surgical Maestro, and a variety of custom systems, spanning surgical manipulation, robotic ultrasound, and endoscopy procedures. We demonstrate the research enabled by this dataset through two foundation models. GR00T-H is the first open foundation vision-language-action model for medical robotics, which is the only evaluated model to achieve full end-to-end task completion on a structured suturing benchmark (25% of trials vs. 0% for all others) and achieves 64% average success across a 29-step ex vivo suturing sequence. We also train Cosmos-H-Surgical-Simulator, the first action-conditioned world model to enable multi-embodiment surgical simulation from a single checkpoint, spanning nine robotic platforms and supporting in silico policy evaluation and synthetic data generation for the medical domain. These results suggest that open, large-scale medical robot data collection can serve as critical infrastructure for the research community, enabling advances in robot learning, world modeling, and beyond.

Problem

• medical robotics
• foundation models
• robotic datasets
• data scarcity
• multi-embodiment

Innovation

• foundation models
• medical robotics
• open dataset
• multi-embodiment
• world modeling