G-SHARP: Gaussian Surgical Hardware Accelerated Real-time Pipeline

2025-12-02

AI Summary

Existing real-time endoscopic surgical scene reconstruction methods rely on non-commercial Gaussian splatting implementations, hindering clinical deployment. To address this, we propose the first commercially compatible, real-time intraoperative 3D reconstruction framework tailored for minimally invasive surgery. Methodologically, we introduce GSplat—the differentiable Gaussian rasterizer—into the surgical reconstruction pipeline for the first time, integrating deformation-aware modeling and robust occlusion handling, alongside real-time optimization algorithms and the NVIDIA Holoscan SDK, enabling low-latency rendering on IGX Orin/Thor edge hardware. Our approach achieves state-of-the-art reconstruction accuracy and real-time performance (>30 FPS) on the EndoNeRF benchmark and demonstrates effective intraoperative visualization on real laparoscopic video sequences. This work significantly advances the clinical translation of Gaussian splatting technology.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Security and Privacy: Data transparency and provenance

Abstract

We propose G-SHARP, a commercially compatible, real-time surgical scene reconstruction framework designed for minimally invasive procedures that require fast and accurate 3D modeling of deformable tissue. While recent Gaussian splatting approaches have advanced real-time endoscopic reconstruction, existing implementations often depend on non-commercial derivatives, limiting deployability. G-SHARP overcomes these constraints by being the first surgical pipeline built natively on the GSplat (Apache-2.0) differentiable Gaussian rasterizer, enabling principled deformation modeling, robust occlusion handling, and high-fidelity reconstructions on the EndoNeRF pulling benchmark. Our results demonstrate state-of-the-art reconstruction quality with strong speed-accuracy trade-offs suitable for intra-operative use. Finally, we provide a Holoscan SDK application that deploys G-SHARP on NVIDIA IGX Orin and Thor edge hardware, enabling real-time surgical visualization in practical operating-room settings.

Problem

• Real-time 3D reconstruction of deformable surgical tissue
• Overcoming non-commercial limitations in endoscopic reconstruction
• Enabling intra-operative deployment on edge hardware platforms

Innovation

• Built on GSplat rasterizer for surgical reconstruction
• Enables real-time deformation modeling and occlusion handling
• Deploys via Holoscan SDK on NVIDIA edge hardware