Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery

2026-07-09

AI Summary

This work addresses the limitations of existing Gaussian splatting methods in robot-assisted minimally invasive surgery, which rely on offline processing and accurate prior camera trajectories, rendering them ineffective under missing priors or noisy conditions. The paper proposes an online 3D Gaussian splatting SLAM framework that jointly optimizes camera poses and deformable anatomical structures from monocular or stereo surgical videos, enabling real-time reconstruction without dependable trajectory priors. Key innovations include deformation initialization from dense 2D point trajectories, statistical trajectory analysis to disentangle camera motion from tissue deformation for drift suppression, and a static-camera interval detection mechanism. Evaluated on the StereoMIS dataset, the method outperforms both existing SLAM approaches and non-SLAM methods that depend on trajectory priors in terms of reconstruction fidelity and trajectory accuracy.

Abstract

Gaussian splatting is the current state-of-the-art for dense, deformable 3D anatomy reconstruction in robot-assisted minimally invasive surgery (RAMIS); however, most pipelines are offline and depend on accurate camera trajectory priors (often from robotic kinematics), limiting applicability when priors are missing or noisy. To address these limitations, we propose Track2Map, an online 3D Gaussian Splatting pipeline that jointly optimizes camera trajectory and 3D deformable scene representation directly from surgical video. Track2Map is therefore capable of robust 3D reconstructions when camera trajectory priors are either absent or noisy, and due to its online nature it effectively works as a Simultaneous Localisation and Mapping (SLAM) method. To stabilize optimization in the presence of tissue motion and ambiguous visual cues, we introduce a track-anchored deformation initialization using dense 2D point tracks. Track statistics are further utilized to disentangle camera motion from scene deformation by detecting static camera periods and reducing drift during incremental mapping. Experiments on StereoMIS show improved reconstruction quality and camera trajectory against competing SLAM methods, as well as compared to non-SLAM methods that utilize camera trajectory priors. The code is available at https://track2map.github.io/.

Problem

• deformable SLAM
• Gaussian splatting
• camera trajectory priors
• robotic surgery
• 3D reconstruction

Innovation

• Gaussian Splatting
• Deformable SLAM
• Motion-Aware Pose Optimization
• Online 3D Reconstruction
• Surgical Robotics