NRGS-SLAM: Monocular Non-Rigid SLAM for Endoscopy via Deformation-Aware 3D Gaussian Splatting

2026-02-19

AI Summary

This work addresses the challenge of tracking drift and reconstruction distortion in monocular SLAM within endoscopic scenes, where continuous soft-tissue deformation violates the rigid-world assumption inherent in conventional approaches. To overcome this limitation, the paper introduces 3D Gaussian splatting into non-rigid SLAM for the first time, proposing a deformable-aware Gaussian representation coupled with a Bayesian self-supervised mechanism that effectively disentangles camera motion from tissue deformation without requiring external deformation labels. Furthermore, a hierarchical tracking and incremental mapping module is designed, integrating geometric priors to mitigate the ill-posed nature of monocular non-rigid reconstruction. Evaluated on multiple public endoscopic datasets, the method reduces camera pose error by up to 50% and achieves photometrically faithful 3D reconstructions that surpass existing approaches in fidelity.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Nonmonotonic Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Visual simultaneous localization and mapping (V-SLAM) is a fundamental capability for autonomous perception and navigation. However, endoscopic scenes violate the rigidity assumption due to persistent soft-tissue deformations, creating a strong coupling ambiguity between camera ego-motion and intrinsic deformation. Although recent monocular non-rigid SLAM methods have made notable progress, they often lack effective decoupling mechanisms and rely on sparse or low-fidelity scene representations, which leads to tracking drift and limited reconstruction quality. To address these limitations, we propose NRGS-SLAM, a monocular non-rigid SLAM system for endoscopy based on 3D Gaussian Splatting. To resolve the coupling ambiguity, we introduce a deformation-aware 3D Gaussian map that augments each Gaussian primitive with a learnable deformation probability, optimized via a Bayesian self-supervision strategy without requiring external non-rigidity labels. Building on this representation, we design a deformable tracking module that performs robust coarse-to-fine pose estimation by prioritizing low-deformation regions, followed by efficient per-frame deformation updates. A carefully designed deformable mapping module progressively expands and refines the map, balancing representational capacity and computational efficiency. In addition, a unified robust geometric loss incorporates external geometric priors to mitigate the inherent ill-posedness of monocular non-rigid SLAM. Extensive experiments on multiple public endoscopic datasets demonstrate that NRGS-SLAM achieves more accurate camera pose estimation (up to 50\% reduction in RMSE) and higher-quality photo-realistic reconstructions than state-of-the-art methods. Comprehensive ablation studies further validate the effectiveness of our key design choices. Source code will be publicly available upon paper acceptance.

Problem

• non-rigid SLAM
• endoscopy
• deformation ambiguity
• monocular vision
• soft-tissue deformation

Innovation

• Deformation-Aware 3D Gaussian Splatting
• Monocular Non-Rigid SLAM
• Bayesian Self-Supervision
• Deformable Tracking
• Geometric Priors