Point Tracking in Surgery--The 2024 Surgical Tattoos in Infrared (STIR) Challenge

2025-03-31

AI Summary

To address the challenge of modeling intraoperative soft-tissue deformation, this paper introduces STIR—the first infrared marker tracking benchmark specifically designed for surgical scenarios. We present a standardized in vivo and ex vivo video dataset with multi-frame temporal trajectories of anatomical landmarks, and propose a dual-dimensional evaluation framework that jointly measures localization accuracy and inference latency. STIR establishes the first quantitative evaluation protocol for intraoperative marker tracking, and we publicly release the STIRMetrics evaluation toolkit, the benchmark dataset (hosted on Zenodo), and baseline implementation code (on GitHub). Eight international teams participated in the challenge, validating the robustness and practicality of the framework. This work sets a new community standard for surgical point tracking, significantly enhancing algorithmic comparability, reproducibility, and clinical translatability under realistic, dynamic intraoperative conditions.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Security and Privacy: Tracking, profiling, and countermeasures against them

Abstract

Understanding tissue motion in surgery is crucial to enable applications in downstream tasks such as segmentation, 3D reconstruction, virtual tissue landmarking, autonomous probe-based scanning, and subtask autonomy. Labeled data are essential to enabling algorithms in these downstream tasks since they allow us to quantify and train algorithms. This paper introduces a point tracking challenge to address this, wherein participants can submit their algorithms for quantification. The submitted algorithms are evaluated using a dataset named surgical tattoos in infrared (STIR), with the challenge aptly named the STIR Challenge 2024. The STIR Challenge 2024 comprises two quantitative components: accuracy and efficiency. The accuracy component tests the accuracy of algorithms on in vivo and ex vivo sequences. The efficiency component tests the latency of algorithm inference. The challenge was conducted as a part of MICCAI EndoVis 2024. In this challenge, we had 8 total teams, with 4 teams submitting before and 4 submitting after challenge day. This paper details the STIR Challenge 2024, which serves to move the field towards more accurate and efficient algorithms for spatial understanding in surgery. In this paper we summarize the design, submissions, and results from the challenge. The challenge dataset is available here: https://zenodo.org/records/14803158 , and the code for baseline models and metric calculation is available here: https://github.com/athaddius/STIRMetrics

Problem

• Develops point tracking for surgical tissue motion analysis
• Evaluates algorithm accuracy and efficiency in surgery
• Provides dataset and metrics for spatial understanding

Innovation

• Infrared surgical tattoos for point tracking
• Accuracy and efficiency evaluation metrics
• Open dataset and baseline models provided