OpenRC: An Open-Source Robotic Colonoscopy Framework for Multimodal Data Acquisition and Autonomy Research

2026-04-04

AI Summary

Existing colonoscopy platforms struggle to systematically investigate the coupled dynamics among operator control, instrument motion, and visual feedback, limiting the reproducibility of closed-loop robotic colonoscopy research. This work proposes OpenRC, an open-source modular framework that retrofits standard endoscopes while preserving clinical workflow, enabling synchronized multimodal data acquisition—including video, control commands, actuator states, and distal pose—through integrated sensors. OpenRC uniquely combines open hardware with a temporally aligned multimodal dataset, facilitating emerging research directions such as vision–language–action learning. Experimental validation confirms system motion consistency, quantifies cross-modal perception latency, and yields a dataset comprising 1,894 teleoperation segments (approximately 19 hours) encompassing navigation, failure modes, and recovery behaviors, thereby establishing a reproducible foundation for surgical autonomy studies.

Abstract

Colorectal cancer screening critically depends on colonoscopy, yet existing platforms offer limited support for systematically studying the coupled dynamics of operator control, instrument motion, and visual feedback. This gap restricts reproducible closed-loop research in robotic colonoscopy, medical imaging, and emerging vision-language-action (VLA) learning paradigms. To address this challenge, we present OpenRC, an open-source modular robotic colonoscopy framework that retrofits conventional scopes while preserving clinical workflow. The framework supports simultaneous recording of video, operator commands, actuation state, and distal tip pose. We experimentally validated motion consistency and quantified cross-modal latency across sensing streams. Using this platform, we collected a multimodal dataset comprising 1,894 teleoperated episodes ~19 hours across 10 structured task variations of routine navigation, failure events, and recovery behaviors. By unifying open hardware and an aligned multimodal dataset, OpenRC provides a reproducible foundation for research in multimodal robotic colonoscopy and surgical autonomy.

Problem

• robotic colonoscopy
• multimodal data acquisition
• surgical autonomy
• vision-language-action learning
• closed-loop research

Innovation

• robotic colonoscopy
• multimodal data acquisition
• open-source framework
• surgical autonomy
• vision-language-action learning