SurgRIPE challenge: Benchmark of Surgical Robot Instrument Pose Estimation

2025-01-06

AI Summary

This study addresses two critical challenges in markerless surgical instrument pose estimation: low accuracy and the absence of standardized evaluation benchmarks. To this end, we introduce SurgVPS—the first open-source surgical video dataset featuring precise, frame-level 6-degree-of-freedom (6DoF) ground-truth annotations—and establish the first international standardized benchmark for markerless instrument pose estimation. Methodologically, our approach integrates multi-view geometric constraints, synthetic surgical video generation with accurate pose supervision, and an end-to-end deep learning framework jointly optimizing 6DoF pose regression and keypoint detection. The proposed method achieves significant improvements in both accuracy and cross-domain generalizability over existing baselines, demonstrating practical feasibility for real-world surgical robot deployment. SurgVPS serves as the official platform for the MICCAI 2023 EndoVis Challenge, enabling rigorous algorithm comparison and advancing vision-driven autonomous surgical systems.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements

Abstract

Accurate instrument pose estimation is a crucial step towards the future of robotic surgery, enabling applications such as autonomous surgical task execution. Vision-based methods for surgical instrument pose estimation provide a practical approach to tool tracking, but they often require markers to be attached to the instruments. Recently, more research has focused on the development of marker-less methods based on deep learning. However, acquiring realistic surgical data, with ground truth instrument poses, required for deep learning training, is challenging. To address the issues in surgical instrument pose estimation, we introduce the Surgical Robot Instrument Pose Estimation (SurgRIPE) challenge, hosted at the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) in 2023. The objectives of this challenge are: (1) to provide the surgical vision community with realistic surgical video data paired with ground truth instrument poses, and (2) to establish a benchmark for evaluating markerless pose estimation methods. The challenge led to the development of several novel algorithms that showcased improved accuracy and robustness over existing methods. The performance evaluation study on the SurgRIPE dataset highlights the potential of these advanced algorithms to be integrated into robotic surgery systems, paving the way for more precise and autonomous surgical procedures. The SurgRIPE challenge has successfully established a new benchmark for the field, encouraging further research and development in surgical robot instrument pose estimation.

Problem

• Algorithm Evaluation
• Unsupervised Learning
• Surgical Robot Precision

Innovation

• SurgRIPE Competition
• Tool Position Prediction
• Unmarked Surgical Tools