Hybrid Mesh-Gaussian Representation for Efficient Indoor Scene Reconstruction

2025-06-08

AI Summary

In indoor scene reconstruction, 3D Gaussian Splatting (3DGS) suffers from low rendering efficiency due to dense Gaussian sampling required in complex texture regions. Method: This paper proposes a mesh-Gaussian hybrid representation: a differentiable textured mesh models large-scale smooth surfaces, while 3D Gaussians capture fine geometry and high-frequency details. We introduce the first joint optimization framework for textured meshes and 3DGS, incorporating warm-start initialization and transmittance-aware supervision to achieve adaptive contribution balancing. Additionally, geometry-guided pruning and refinement strategies enhance optimization stability and convergence. Results: Experiments demonstrate that our method maintains high rendering quality while significantly improving frame rate (FPS). The number of Gaussian primitives is reduced by 30–50%, enabling the first real-time, high-fidelity indoor scene reconstruction via hybrid representation.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Computer Vision: 3D Computer Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

3D Gaussian splatting (3DGS) has demonstrated exceptional performance in image-based 3D reconstruction and real-time rendering. However, regions with complex textures require numerous Gaussians to capture significant color variations accurately, leading to inefficiencies in rendering speed. To address this challenge, we introduce a hybrid representation for indoor scenes that combines 3DGS with textured meshes. Our approach uses textured meshes to handle texture-rich flat areas, while retaining Gaussians to model intricate geometries. The proposed method begins by pruning and refining the extracted mesh to eliminate geometrically complex regions. We then employ a joint optimization for 3DGS and mesh, incorporating a warm-up strategy and transmittance-aware supervision to balance their contributions seamlessly.Extensive experiments demonstrate that the hybrid representation maintains comparable rendering quality and achieves superior frames per second FPS with fewer Gaussian primitives.

Problem

• Efficient 3D reconstruction of texture-rich indoor scenes
• Balancing rendering speed and quality in complex geometries
• Reducing Gaussian primitives while maintaining high FPS

Innovation

• Hybrid 3DGS and textured mesh representation
• Mesh pruning and refinement for complex regions
• Joint optimization with warm-up strategy