Decoding the Surgical Scene: A Scoping Review of Scene Graphs in Surgery

2025-09-25

AI Summary

This paper addresses the “data gap” in surgical scene graph (SG) research—where endoscopic analysis (e.g., triplet detection) relies on real 2D videos, while exoscopic 4D modeling heavily depends on synthetic data—hindering cross-view translation. Guided by the PRISMA-ScR framework, we conduct the first systematic review to expose this data fragmentation and propose SGs as a unifying semantic bridge between surgical analysis and generative tasks. By integrating graph neural networks, domain-specific foundation models, and vision-language large models, we establish SGs’ central role in surgical workflow recognition, real-time safety monitoring, and controllable simulation. Our synthesis demonstrates that SGs significantly enhance the interpretability, generalizability, and clinical adaptability of intelligent surgical systems—bridging critical gaps between perception, reasoning, and generation across intra- and extra-operative modalities.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Scene Analysis & Understanding
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Scene graphs (SGs) provide structured relational representations crucial for decoding complex, dynamic surgical environments. This PRISMA-ScR-guided scoping review systematically maps the evolving landscape of SG research in surgery, charting its applications, methodological advancements, and future directions. Our analysis reveals rapid growth, yet uncovers a critical 'data divide': internal-view research (e.g., triplet recognition) almost exclusively uses real-world 2D video, while external-view 4D modeling relies heavily on simulated data, exposing a key translational research gap. Methodologically, the field has advanced from foundational graph neural networks to specialized foundation models that now significantly outperform generalist large vision-language models in surgical contexts. This progress has established SGs as a cornerstone technology for both analysis, such as workflow recognition and automated safety monitoring, and generative tasks like controllable surgical simulation. Although challenges in data annotation and real-time implementation persist, they are actively being addressed through emerging techniques. Surgical SGs are maturing into an essential semantic bridge, enabling a new generation of intelligent systems to improve surgical safety, efficiency, and training.

Problem

• Addressing the data divide between real-world 2D video and simulated 4D data in surgical scene analysis
• Advancing from basic graph neural networks to specialized foundation models for surgical contexts
• Developing surgical scene graphs as semantic bridges for intelligent systems to enhance safety and training

Innovation

• Specialized foundation models outperform generalist vision-language models
• Scene graphs enable workflow recognition and automated safety monitoring
• Graph neural networks provide structured surgical environment representations