ReSW-VL: Representation Learning for Surgical Workflow Analysis Using Vision-Language Model
Convolutional neural networks (CNNs) for surgical phase recognition suffer from heavy reliance on strong supervision and poor generalizability across domains. Method: This paper introduces the first vision–language collaborative representation framework for surgical workflow analysis, integrating the CLIP image encoder with prompt learning. It further incorporates fine-grained temporal modeling and multi-dataset transfer training to alleviate domain adaptation bottlenecks. Contribution/Results: Evaluated on three public surgical phase recognition benchmarks, the proposed method achieves average accuracy improvements of 3.2–5.8 percentage points over state-of-the-art CNN-based temporal models. It significantly enhances intraoperative video understanding and provides robust, semantically grounded representations for real-time clinical decision support, operating room resource scheduling, and surgical skill assessment.