Sterilizable Scene Graph Generation for Operating Rooms

2026-08-17

AI Summary

This study addresses the challenges of parameter redundancy, deployment difficulties, and privacy risks in operating room scene graph generation by proposing the SG-NCA framework. This approach introduces a novel paradigm integrating Neural Cellular Automata (NCA) for scene graph generation and structured representation learning, combined with multi-class segmentation and a lightweight relation predictor. Experimental results demonstrate that SG-NCA achieves performance comparable to mainstream baselines while reducing model parameters by 55 times. Consequently, it enables successful deployment on fanless edge devices, effectively satisfying sterile environment requirements and ensuring data privacy. These findings establish SG-NCA as a viable solution for lightweight medical AI applications, offering a new pathway for secure and efficient intraoperative analysis without compromising accuracy or safety standards in clinical settings.

Abstract

Scene graph generation from surgical video enables a holistic and structured understanding of surgical scenes by modeling objects and their semantic relationships. Despite recent advances, state-of-the-art approaches rely on large, parameter-heavy deep learning models that are impractical for deployment in the operating room (OR) due to hardware footprint, hygiene constraints, latency, and data privacy concerns. To the best of our knowledge, this is the first scene graph generation method built on NCAs and the first NCA framework capable of learning structured representations. We introduce SG-NCA, a lightweight scene graph generation framework based on Neural Cellular Automata (NCA), designed for inference in fanless devices critical for OR hygiene protocols. SG-NCA is the first scene graph generation combining NCA-based multiclass segmentation for efficient object detection and feature extraction with a lightweight relation predictor. We evaluate SG-NCA on videos of cataract surgery and cholecystectomy, demonstrating performance comparable to established baselines while requiring 55x fewer parameters. We showcase deployment on fanless edge devices better suited for the OR and demonstrate downstream applications such as surgical video captioning, highlighting SG-NCA's potential for affordable, privacy-preserving, and OR-ready intraoperative scene understanding.

Problem

• Scene Graph Generation
• Operating Room
• Edge Deployment
• Lightweight Model
• Surgical Video Understanding

Innovation

• Neural Cellular Automata
• Scene Graph Generation
• Lightweight Framework
• Fanless Edge Deployment
• Surgical Scene Understanding