MoonSplat: Monocular Online Gaussian Splatting with Sim(3) Global Optimization

2026-06-16

AI Summary

This work addresses the challenges of fragile camera pose estimation, lack of global optimization, and low efficiency in large-scale monocular online 3D reconstruction by proposing a voxelized 3D Gaussian Splatting framework integrated with Sim(3) global optimization. For the first time, Sim(3)-based global pose refinement is incorporated into online 3D Gaussian Splatting, combined with a voxelized scene representation and a color residual learning strategy. This integration significantly enhances both pose accuracy and rendering quality while maintaining real-time performance. Extensive experiments demonstrate state-of-the-art results across diverse indoor and outdoor datasets. Furthermore, the method has been successfully deployed in a real-world drone-based active reconstruction system, validating its effectiveness and generalization capability.

Abstract

Online 3D reconstruction from monocular image sequences is a challenging and ongoing research topic. 3D Gaussian Splatting (3DGS), leveraging its high-quality real-time rendering capability, empowers online 3D reconstruction to represent dense scenes with enhanced expressiveness, and thus holds great promise for a wide range of applications such as robotics and AR/VR. However, existing online 3DGS methods still suffer from some key challenges: fragile camera pose estimation due to the lack of global optimization, and low optimization efficiency in large-scale or long-sequence scenarios. To address these issues, we propose a robust and efficient online voxelized 3DGS reconstruction framework integrated with global $\text{Sim}(3)$ optimization, which enables reliable camera tracking and efficient global loop closure for both camera poses and voxelized 3DGS. To accelerate the convergence of the voxelized 3DGS, we further introduce a color residual learning strategy, which not only boosts optimization speed but also enhances rendering quality. Extensive experiments on diverse indoor and outdoor datasets demonstrate that our method achieves state-of-the-art performance in both camera pose estimation accuracy and rendering quality, while retaining real-time efficiency. Additionally, we develop and deploy a real-world UAV-based active reconstruction system grounded on our proposed method, validating its robustness and generalizability for practical online 3D reconstruction tasks. Our code and data are available at https://github.com/TrickyGo/MoonSplat.

Problem

• online 3D reconstruction
• monocular
• 3D Gaussian Splatting
• camera pose estimation
• global optimization

Innovation

• Gaussian Splatting
• Sim(3) Global Optimization
• Online 3D Reconstruction
• Monocular SLAM
• Voxelized Representation