ExploreGS: a vision-based low overhead framework for 3D scene reconstruction

2025-05-14

AI Summary

To address the challenge of deploying computationally expensive LiDAR-based 3D reconstruction on resource-constrained UAV platforms, this paper proposes a lightweight, end-to-end, vision-only 3D Gaussian Splatting (3DGS) reconstruction framework. Methodologically, it jointly optimizes active scene exploration with 3DGS training and introduces a Bag-of-Words (BoW) feature indexing mechanism to enable onboard real-time, memory-efficient online training. The framework operates solely on RGB inputs—requiring no depth supervision or IMU priors—and executes the full 3DGS training pipeline directly on embedded devices. Experimentally, it achieves state-of-the-art reconstruction quality while reducing computational overhead by over 60%, thereby significantly alleviating the deployment bottleneck of 3DGS on edge devices.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper proposes a low-overhead, vision-based 3D scene reconstruction framework for drones, named ExploreGS. By using RGB images, ExploreGS replaces traditional lidar-based point cloud acquisition process with a vision model, achieving a high-quality reconstruction at a lower cost. The framework integrates scene exploration and model reconstruction, and leverags a Bag-of-Words(BoW) model to enable real-time processing capabilities, therefore, the 3D Gaussian Splatting (3DGS) training can be executed on-board. Comprehensive experiments in both simulation and real-world environments demonstrate the efficiency and applicability of the ExploreGS framework on resource-constrained devices, while maintaining reconstruction quality comparable to state-of-the-art methods.

Problem

• Vision-based 3D scene reconstruction for drones
• Replacing lidar with RGB images for cost efficiency
• Real-time on-board processing for resource-constrained devices

Innovation

• Vision-based 3D reconstruction replaces lidar
• Integrates scene exploration and model reconstruction
• Enables real-time on-board 3DGS training