Monocular Endoscopic Tissue 3D Reconstruction with Multi-Level Geometry Regularization

2025-06-30

IEEE International Joint Conference on Neural Network

AI Summary

Real-time, smooth 3D reconstruction of highly deformable soft tissues from monocular endoscopic views remains challenging. This work proposes a novel approach that integrates signed distance field (SDF)-guided meshes with 3D Gaussian splatting, leveraging multi-level geometric regularization—combining local rigidity constraints and global non-rigid deformation modeling—to achieve high-quality surface reconstruction and real-time rendering while preserving physical plausibility. The method significantly outperforms existing techniques, delivering enhanced geometric fidelity and richer texture detail, making it well-suited for demanding applications such as robot-assisted surgery where both accuracy and efficiency are critical.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Reconstructing deformable endoscopic tissues is crucial for achieving robot-assisted surgery. However, 3D Gaussian Splatting-based approaches encounter challenges in achieving consistent tissue surface reconstruction, while existing NeRF-based methods lack real-time rendering capabilities. In pursuit of both smooth deformable surfaces and real-time rendering, we introduce a novel approach based on 3D Gaussian Splatting. Specifically, we introduce surface-aware reconstruction, initially employing a Sign Distance Field-based method to construct a mesh, subsequently utilizing this mesh to constrain the Gaussian Splatting reconstruction process. Furthermore, to ensure the generation of physically plausible deformations, we incorporate local rigidity and global non-rigidity restrictions to guide Gaussian deformation, tailored for the highly deformable nature of soft endoscopic tissue. Based on 3D Gaussian Splatting, our proposed method delivers a fast rendering process and smooth surface appearances. Quantitative and qualitative analysis against alternative methodologies shows that our approach achieves solid reconstruction quality in both textures and geometries.

Problem

• monocular endoscopy
• deformable tissue
• 3D reconstruction
• real-time rendering
• surface consistency

Innovation

• 3D Gaussian Splatting
• Surface-aware Reconstruction
• Geometry Regularization
• Deformable Tissue Modeling
• Endoscopic 3D Reconstruction