DeGenseGS: Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting

2026-07-06

AI Summary

This work addresses the severe misalignment between semantic information and anatomical geometry in surgical scenes, a challenge that existing methods struggle to overcome due to their coupled modeling of semantics and geometry, which hinders efficient and text-promptable 4D reconstruction. To resolve this, the authors propose a novel paradigm within the 4D Gaussian splatting framework that decouples semantic evolution from geometric deformation while synchronizing their dynamics through a shared motion latent variable. Key innovations include a HexPlane spatiotemporal entanglement module, a rasterization-native semantic extraction mechanism, and a hyperspherical latent space angular alignment strategy, collectively enhancing semantic-geometry consistency without compromising spatial continuity. Experiments demonstrate state-of-the-art performance on CholecSeg8k and EndoVis18, significantly improving geometric completeness and segmentation robustness under large deformations.

Abstract

Real-time, text-promptable 4D reconstruction is indispensable for autonomous surgical interaction. Severe misalignment between semantic meaning and physical anatomy still persists, largely because existing solutions integrate Vision-Language Models into deformable fields via a rigid coupling scheme that tightly binds semantic features to geometric warping. In this paper, we propose DeGenseGS, Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting, a novel framework that independently models semantic evolution and geometric deformation. Specifically, we propose a HexPlane-based spatiotemporal entanglement module that uses shared kinematic latents to synchronize semantic mutations with scene dynamics, while explicitly disentangling semantic updates from geometric deformation. To further ensure robustness against reconstruction artifacts, we devise a Rasterization-Native Semantic Extraction mechanism that infers semantics from topologically continuous feature maps. Additionally, we incorporate an angular-aligned optimization strategy that conforms to the native hyperspherical latent space, thereby preventing semantic distortion. Extensive evaluations on the CholecSeg8k and EndoVis18 datasets demonstrate that DeGenseGS achieves state-of-the-art performance. Our framework yields enhanced geometric completeness and robust semantic-anatomic alignment, enabling spatially continuous segmentation despite drastic tissue deformation and topological transitions.

Problem

• 4D reconstruction
• surgical scene understanding
• semantic-geometric misalignment
• Gaussian Splatting
• vision-language models

Innovation

• Geometric-Semantic Decoupling
• 4D Gaussian Splatting
• HexPlane
• Rasterization-Native Semantic Extraction
• Angular-Aligned Optimization