ProstaTD: A Large-scale Multi-source Dataset for Structured Surgical Triplet Detection

2025-06-01

AI Summary

Existing surgical triplet detection datasets suffer from coarse spatial annotations, temporally ungrounded labels lacking clinical justification, and poor generalizability due to single-center collection. To address these limitations, we introduce RAP-TRIP—the first large-scale, multi-center surgical triplet dataset specifically designed for robot-assisted prostatectomy—comprising 21 cross-institutional procedures, 60,500 video frames, and 165,600 high-fidelity spatiotemporal triplet annotations. RAP-TRIP pioneers clinically grounded temporal boundaries (defined by expert surgeons) and pixel-accurate spatial bounding boxes, established via a surgeon-led, dual-track iterative annotation protocol. As the largest, most diverse, and clinically validated surgical triplet benchmark to date, RAP-TRIP substantially enhances model generalizability across institutions and surgical workflows. It serves as a foundational resource for developing trustworthy surgical AI systems and standardizing procedural training.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

Surgical triplet detection has emerged as a pivotal task in surgical video analysis, with significant implications for performance assessment and the training of novice surgeons. However, existing datasets such as CholecT50 exhibit critical limitations: they lack precise spatial bounding box annotations, provide inconsistent and clinically ungrounded temporal labels, and rely on a single data source, which limits model generalizability.To address these shortcomings, we introduce ProstaTD, a large-scale, multi-institutional dataset for surgical triplet detection, developed from the technically demanding domain of robot-assisted prostatectomy. ProstaTD offers clinically defined temporal boundaries and high-precision bounding box annotations for each structured triplet action. The dataset comprises 60,529 video frames and 165,567 annotated triplet instances, collected from 21 surgeries performed across multiple institutions, reflecting a broad range of surgical practices and intraoperative conditions. The annotation process was conducted under rigorous medical supervision and involved more than 50 contributors, including practicing surgeons and medically trained annotators, through multiple iterative phases of labeling and verification. ProstaTD is the largest and most diverse surgical triplet dataset to date, providing a robust foundation for fair benchmarking, the development of reliable surgical AI systems, and scalable tools for procedural training.

Problem

• Lack precise spatial bounding box annotations in existing datasets
• Inconsistent and clinically ungrounded temporal labels in current datasets
• Limited model generalizability due to single data source reliance

Innovation

• Large-scale multi-institutional surgical dataset
• Clinically defined temporal and spatial annotations
• Diverse data from 21 surgeries for robustness