Deformable Gaussian Splatting for Efficient and High-Fidelity Reconstruction of Surgical Scenes

2025-01-02

AI Summary

Addressing the challenges of modeling irreversible tissue deformations (e.g., cutting, tearing) in surgical scenarios and the lack of hierarchical acceleration in existing Gaussian radiance field methods—leading to slow reconstruction—this paper proposes EH-SurGS, a deformable Gaussian rasterization framework tailored for surgical dynamic reconstruction. Our method extends 3D Gaussian Splatting by integrating a learnable deformation field, a novel Gaussian lifetime modeling mechanism that unifies reversible and irreversible deformations, and an adaptive motion-aware hierarchical scheduling strategy that separates static and dynamic regions to jointly optimize accuracy and efficiency. Evaluated on multiple surgical datasets, EH-SurGS achieves a 2.1 dB PSNR improvement and a 37% rendering speedup over baseline methods, significantly enhancing both reconstruction fidelity and real-time performance.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Computer Vision: 3D Computer Vision

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Efficient and high-fidelity reconstruction of deformable surgical scenes is a critical yet challenging task. Building on recent advancements in 3D Gaussian splatting, current methods have seen significant improvements in both reconstruction quality and rendering speed. However, two major limitations remain: (1) difficulty in handling irreversible dynamic changes, such as tissue shearing, which are common in surgical scenes; and (2) the lack of hierarchical modeling for surgical scene deformation, which reduces rendering speed. To address these challenges, we introduce EH-SurGS, an efficient and high-fidelity reconstruction algorithm for deformable surgical scenes. We propose a deformation modeling approach that incorporates the life cycle of 3D Gaussians, effectively capturing both regular and irreversible deformations, thus enhancing reconstruction quality. Additionally, we present an adaptive motion hierarchy strategy that distinguishes between static and deformable regions within the surgical scene. This strategy reduces the number of 3D Gaussians passing through the deformation field, thereby improving rendering speed. Extensive experiments demonstrate that our method surpasses existing state-of-the-art approaches in both reconstruction quality and rendering speed. Ablation studies further validate the effectiveness and necessity of our proposed components. We will open-source our code upon acceptance of the paper.

Problem

• Surgical Process Reconstruction
• Irreversible Deformation
• Hierarchical Processing

Innovation

• EH-SurGS Algorithm
• Real-time Visualization
• Hierarchical Deformation Handling