🤖 AI Summary
Existing online 3D Gaussian Splatting (3DGS) methods rely solely on sparse keyframes, resulting in incomplete scene coverage, geometric incompleteness, and a fundamental trade-off between modeling fidelity and scalability under real-time constraints.
Method: We propose an online, end-to-end 3DGS framework for pure RGB video streams. It introduces a reconstruction-quality-driven adaptive view selection mechanism that jointly optimizes keyframes and dynamically selected non-keyframes. Integrating online SLAM, multi-view stereo matching, and incremental 3DGS training, the framework enables real-time, dynamic identification and efficient incorporation of informative non-keyframes.
Contribution/Results: Evaluated on complex dynamic outdoor scenes, our method significantly improves reconstruction completeness and geometric accuracy. It outperforms state-of-the-art approaches in robustness and efficiency, achieving—for the first time—high-fidelity, full-coverage online 3DGS reconstruction under strict real-time constraints.
📝 Abstract
This study addresses the challenge of generating online 3D Gaussian Splatting (3DGS) models from RGB-only frames. Previous studies have employed dense SLAM techniques to estimate 3D scenes from keyframes for 3DGS model construction. However, these methods are limited by their reliance solely on keyframes, which are insufficient to capture an entire scene, resulting in incomplete reconstructions. Moreover, building a generalizable model requires incorporating frames from diverse viewpoints to achieve broader scene coverage. However, online processing restricts the use of many frames or extensive training iterations. Therefore, we propose a novel method for high-quality 3DGS modeling that improves model completeness through adaptive view selection. By analyzing reconstruction quality online, our approach selects optimal non-keyframes for additional training. By integrating both keyframes and selected non-keyframes, the method refines incomplete regions from diverse viewpoints, significantly enhancing completeness. We also present a framework that incorporates an online multi-view stereo approach, ensuring consistency in 3D information throughout the 3DGS modeling process. Experimental results demonstrate that our method outperforms state-of-the-art methods, delivering exceptional performance in complex outdoor scenes.