Surgical Anatomy Recognition with Context Learning using Foundation Representations

2026-06-20

AI Summary

This study addresses the challenges of anatomical structure recognition in minimally invasive surgery, which are hindered by scarce annotated data and the bias of existing methods toward natural scenes. To overcome these limitations, the authors introduce ATLAS-120k, a large-scale semantic segmentation dataset comprising 120,000 video frames spanning 14 distinct surgical procedures. They propose the context-aware ATLAS model, which integrates foundation model embeddings with lightweight temporal reasoning, leveraging surgical type, procedural phase, and short-term visual memory to enhance both accuracy and temporal consistency. An innovative, scalable annotation pipeline combining expert labeling, automated propagation, and surgeon validation is also developed. The proposed approach achieves real-time inference while significantly improving segmentation performance, thereby laying a foundation for surgical navigation and clinical decision-support systems.

Abstract

Accurate recognition of anatomical structures is essential for safe and effective minimally invasive surgery (MIS), yet it remains underexplored in surgical computer vision due to limited annotated data and methods tailored primarily to natural scenes. In this work, we present a combined dataset and model framework to advance anatomy-aware perception in MIS. First, we introduce ATLAS-120k, a large-scale clip-level semantic segmentation dataset comprising over 120,000 annotated frames from 100 surgical videos spanning 14 procedures and multiple modalities, including laparoscopic and robot-assisted surgery. The dataset captures substantial procedural variability and was created using a scalable annotation pipeline that integrates expert manual labeling, automated propagation, iterative refinement, and surgeon verification to ensure high-quality annotations. Second, we propose ATLAS (Anatomy Recognition with Context Learning using Foundation Representations), a video semantic segmentation model specifically designed for surgical anatomy recognition. Unlike conventional approaches that emphasize object tracking, ATLAS leverages foundation-model embeddings together with lightweight temporal reasoning to incorporate contextual cues such as procedure type, surgical phase, and short-term visual memory. This design enables temporally consistent and accurate predictions while maintaining real-time feasibility. Together, the dataset and model establish a practical foundation for robust surgical scene understanding and support the development of clinically applicable guidance systems for minimally invasive surgery. The models, dataset annotations and annotation platform are publicly available at: https://github.com/TimJaspers0801/ATLAS.

Problem

• surgical anatomy recognition
• minimally invasive surgery
• semantic segmentation
• limited annotated data
• surgical computer vision

Innovation

• foundation representations
• surgical anatomy recognition
• context learning
• semantic segmentation
• minimally invasive surgery