Towards Controllable Video Synthesis of Routine and Rare OR Events

2026-02-24

AI Summary

This work addresses the scarcity of large-scale real-world surgical video datasets containing rare, safety-critical, or atypical events—a key bottleneck in advancing intelligent operating room systems. To overcome this limitation, we propose the first controllable video generation framework based on geometric abstraction, which integrates geometric representations, conditional control mechanisms, and fine-tuned video diffusion models to efficiently synthesize high-fidelity videos of both routine and rare surgical scenarios, including counterfactual cases and aseptic violations. Experimental results demonstrate that our method outperforms existing baselines in generating routine surgical events, achieving lower FVD and LPIPS scores alongside higher SSIM and PSNR values. Furthermore, detection models trained on our synthetic data attain a recall of 70.13% on near-miss aseptic violation events, effectively mitigating the constraints imposed by real-data scarcity.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Purpose: Curating large-scale datasets of operating room (OR) workflow, encompassing rare, safety-critical, or atypical events, remains operationally and ethically challenging. This data bottleneck complicates the development of ambient intelligence for detecting, understanding, and mitigating rare or safety-critical events in the OR. Methods: This work presents an OR video diffusion framework that enables controlled synthesis of rare and safety-critical events. The framework integrates a geometric abstraction module, a conditioning module, and a fine-tuned diffusion model to first transform OR scenes into abstract geometric representations, then condition the synthesis process, and finally generate realistic OR event videos. Using this framework, we also curate a synthetic dataset to train and validate AI models for detecting near-misses of sterile-field violations. Results: In synthesizing routine OR events, our method outperforms off-the-shelf video diffusion baselines, achieving lower FVD/LPIPS and higher SSIM/PSNR in both in- and out-of-domain datasets. Through qualitative results, we illustrate its ability for controlled video synthesis of counterfactual events. An AI model trained and validated on the generated synthetic data achieved a RECALL of 70.13% in detecting near safety-critical events. Finally, we conduct an ablation study to quantify performance gains from key design choices. Conclusion: Our solution enables controlled synthesis of routine and rare OR events from abstract geometric representations. Beyond demonstrating its capability to generate rare and safety-critical scenarios, we show its potential to support the development of ambient intelligence models.

Problem

• operating room
• rare events
• safety-critical events
• dataset curation
• ambient intelligence

Innovation

• controlled video synthesis
• surgical workflow
• diffusion model
• geometric abstraction
• rare event generation