SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Video

2026-02-25

AI Summary

Current AI systems struggle to integrate visual, anatomical, and procedural context during critical phases of minimally invasive surgery to dynamically identify safe operating zones, and they lack explicit modeling of dependencies across intraoperative reasoning stages. To address this, this work proposes SurGo-R1, a two-stage architecture that first recognizes the surgical phase and then generates context-aware reasoning alongside coordinates of safe operating regions, optimized via reinforcement learning from human feedback (RLHF). The study also introduces ResGo, the first benchmark featuring multidimensional clinical reasoning annotations—including surgical phase, exposure quality, next-action intent, and risk alerts—and adopts a novel evaluation paradigm where any phase misclassification constitutes task failure. Experiments show that the method achieves 76.6% phase accuracy, 32.7 mIoU, and 54.8% hard-accuracy on unseen surgical videos, outperforming general-purpose vision-language models by a factor of 6.6.

Technology Category

• Natural Language Processing: Safety and Robustness
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Minimally invasive surgery has dramatically improved patient operative outcomes, yet identifying safe operative zones remains challenging in critical phases, requiring surgeons to integrate visual cues, procedural phase, and anatomical context under high cognitive load. Existing AI systems offer binary safety verification or static detection, ignoring the phase-dependent nature of intraoperative reasoning. We introduce ResGo, a benchmark of laparoscopic frames annotated with Go Zone bounding boxes and clinician-authored rationales covering phase, exposure quality reasoning, next action and risk reminder. We introduce evaluation metrics that treat correct grounding under incorrect phase as failures, revealing that most vision-language models cannot handle such tasks and perform poorly. We then present SurGo-R1, a model optimized via RLHF with a multi-turn phase-then-go architecture where the model first identifies the surgical phase, then generates reasoning and Go Zone coordinates conditioned on that context. On unseen procedures, SurGo-R1 achieves 76.6% phase accuracy, 32.7 mIoU, and 54.8% hardcore accuracy, a 6.6$\times$ improvement over the mainstream generalist VLMs. Code, model and benchmark will be available at https://github.com/jinlab-imvr/SurGo-R1

Problem

• operative zone
• contextual reasoning
• surgical video
• phase-dependent reasoning
• minimally invasive surgery

Innovation

• contextual reasoning
• surgical video analysis
• phase-aware modeling
• vision-language model
• RLHF