XiCAD: Camera Activation Detection in the Da Vinci Xi User Interface

2025-11-25

AI Summary

Automatic identification of endoscopic activation status in da Vinci Xi robotic surgery videos remains challenging due to ambiguous visual cues and frequent multi-camera configurations. Method: We propose a lightweight, high-accuracy UI-aware detection method based on a ResNet18 backbone, trained for end-to-end binary classification. It precisely localizes the camera tile within the native Xi user interface and infers activation state from its visual indicators, outputting real-time camera motion metadata. Contribution/Results: This is the first robust, UI-native activation detector for the Xi system, eliminating false positives from concurrent cameras. Evaluated on >70,000 frames from real surgical videos, it achieves F1 scores of 0.993–1.000, with accurate, non-redundant localization. The method delivers reliable, structured metadata—enabling downstream applications including intraoperative instrument tracking, surgical skill assessment, and automated camera control—thereby addressing a critical gap in da Vinci video metadata parsing.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Purpose: Robot-assisted minimally invasive surgery relies on endoscopic video as the sole intraoperative visual feedback. The DaVinci Xi system overlays a graphical user interface (UI) that indicates the state of each robotic arm, including the activation of the endoscope arm. Detecting this activation provides valuable metadata such as camera movement information, which can support downstream surgical data science tasks including tool tracking, skill assessment, or camera control automation. Methods: We developed a lightweight pipeline based on a ResNet18 convolutional neural network to automatically identify the position of the camera tile and its activation state within the DaVinci Xi UI. The model was fine-tuned on manually annotated data from the SurgToolLoc dataset and evaluated across three public datasets comprising over 70,000 frames. Results: The model achieved F1-scores between 0.993 and 1.000 for the binary detection of active cameras and correctly localized the camera tile in all cases without false multiple-camera detections. Conclusion: The proposed pipeline enables reliable, real-time extraction of camera activation metadata from surgical videos, facilitating automated preprocessing and analysis for diverse downstream applications. All code, trained models, and annotations are publicly available.

Problem

• Detecting camera activation state in DaVinci Xi surgical interface
• Automating extraction of camera movement metadata from endoscopic videos
• Enabling automated preprocessing for surgical data science applications

Innovation

• Lightweight ResNet18 pipeline for camera activation detection
• Fine-tuned model on SurgToolLoc surgical dataset annotations
• Real-time extraction of endoscopic camera metadata