LighthouseGS: Indoor Structure-aware 3D Gaussian Splatting for Panorama-Style Mobile Captures

2025-07-08

AI Summary

In texture-poor indoor scenes, handheld panoramic capture with mobile phones suffers from inaccurate pose estimation, unstable geometric reconstruction, and degraded 3D Gaussian Splatting (3DGS) rendering—primarily due to rotation-dominant motion and narrow-baseline stereo constraints. To address this, we propose a robust panoramic 3DGS reconstruction framework tailored for mobile devices. Our key contributions are: (1) a lighthouse-inspired planar skeleton initialization that leverages indoor planar structural priors to yield geometrically consistent initial point clouds; (2) a planar-structure-guided stable pruning and joint geometric-photometric correction mechanism to suppress motion drift and auto-exposure artifacts; and (3) an adaptive optimization strategy integrating monocular depth, IMU pose priors, and planar geometric constraints. Evaluated on both real and synthetic indoor scenes, our method achieves photorealistic novel-view synthesis, significantly outperforming existing mobile-oriented 3DGS approaches, while enabling high-fidelity panoramic rendering and seamless virtual object insertion.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Localization, Mapping, and Navigation
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Recent advances in 3D Gaussian Splatting (3DGS) have enabled real-time novel view synthesis (NVS) with impressive quality in indoor scenes. However, achieving high-fidelity rendering requires meticulously captured images covering the entire scene, limiting accessibility for general users. We aim to develop a practical 3DGS-based NVS framework using simple panorama-style motion with a handheld camera (e.g., mobile device). While convenient, this rotation-dominant motion and narrow baseline make accurate camera pose and 3D point estimation challenging, especially in textureless indoor scenes. To address these challenges, we propose LighthouseGS, a novel framework inspired by the lighthouse-like sweeping motion of panoramic views. LighthouseGS leverages rough geometric priors, such as mobile device camera poses and monocular depth estimation, and utilizes the planar structures often found in indoor environments. We present a new initialization method called plane scaffold assembly to generate consistent 3D points on these structures, followed by a stable pruning strategy to enhance geometry and optimization stability. Additionally, we introduce geometric and photometric corrections to resolve inconsistencies from motion drift and auto-exposure in mobile devices. Tested on collected real and synthetic indoor scenes, LighthouseGS delivers photorealistic rendering, surpassing state-of-the-art methods and demonstrating the potential for panoramic view synthesis and object placement.

Problem

• Achieving high-fidelity 3D rendering with simple panorama-style mobile captures
• Overcoming challenges in camera pose and 3D point estimation in textureless indoor scenes
• Enhancing geometry and optimization stability for indoor structure-aware 3D Gaussian Splatting

Innovation

• Uses lighthouse-like sweeping motion for 3DGS
• Leverages geometric priors and planar structures
• Introduces plane scaffold assembly initialization