VarSplat: Uncertainty-aware 3D Gaussian Splatting for Robust RGB-D SLAM

2026-03-10

AI Summary

This work addresses the susceptibility of existing 3D Gaussian splatting-based SLAM methods to pose drift and global misalignment in low-texture, transparent, or highly reflective regions, primarily due to the neglect of measurement reliability. To overcome this limitation, we introduce explicit uncertainty modeling into the framework for the first time. By learning per-Gaussian appearance variances and integrating the law of total variance with alpha compositing, our method efficiently generates pixel-wise uncertainty maps within a single differentiable rasterization pass. These uncertainty maps are leveraged to guide camera tracking, submap registration, and loop closure. Extensive experiments on Replica, TUM-RGBD, ScanNet, and ScanNet++ demonstrate that our approach significantly enhances robustness and achieves state-of-the-art or comparable performance against dense RGB-D SLAM methods in terms of tracking accuracy, mapping quality, and novel view synthesis.

Technology Category

• Reasoning under Uncertainty: Uncertainty Representations
• Machine Learning: Calibration & Uncertainty Quantification
• Planning, Routing, and Scheduling: Planning under Uncertainty

Application Category

• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Simultaneous Localization and Mapping (SLAM) with 3D Gaussian Splatting (3DGS) enables fast, differentiable rendering and high-fidelity reconstruction across diverse real-world scenes. However, existing 3DGS-SLAM approaches handle measurement reliability implicitly, making pose estimation and global alignment susceptible to drift in low-texture regions, transparent surfaces, or areas with complex reflectance properties. To this end, we introduce VarSplat, an uncertainty-aware 3DGS-SLAM system that explicitly learns per-splat appearance variance. By using the law of total variance with alpha compositing, we then render differentiable per-pixel uncertainty map via efficient, single-pass rasterization. This map guides tracking, submap registration, and loop detection toward focusing on reliable regions and contributes to more stable optimization. Experimental results on Replica (synthetic) and TUM-RGBD, ScanNet, and ScanNet++ (real-world) show that VarSplat improves robustness and achieves competitive or superior tracking, mapping, and novel view synthesis rendering compared to existing studies for dense RGB-D SLAM.

Problem

• 3D Gaussian Splatting
• RGB-D SLAM
• measurement uncertainty
• pose drift
• robustness

Innovation

• Uncertainty-aware
• 3D Gaussian Splatting
• RGB-D SLAM
• Differentiable Rendering
• Variance Modeling