HRGS: Hierarchical Gaussian Splatting for Memory-Efficient High-Resolution 3D Reconstruction

2025-06-17

AI Summary

To address the GPU memory explosion and poor scalability of 3D Gaussian Splatting (3DGS) in high-resolution scenes, this paper proposes a hierarchical block-wise optimization framework. It first constructs a global coarse Gaussian representation, then partitions the scene into spatial blocks guided jointly by Gaussian distribution and observation coverage, enabling coarse-to-fine inter-block Gaussian fusion. We introduce two key innovations: Importance-Driven Gaussian Pruning (IDGP) and normal-prior distillation, achieving seamless high-fidelity reconstruction under strict memory constraints. The method integrates hierarchical optimization, spatially adaptive block partitioning, multi-scale observation cropping, and geometry-aware prior enhancement. Evaluated on three major benchmarks, our approach achieves state-of-the-art performance in novel-view synthesis and surface reconstruction, reduces GPU memory consumption by 42–68%, accelerates convergence by 1.8×, and—critically—enables high-quality high-resolution 3D reconstruction on a single consumer-grade GPU (≤16 GB VRAM) for the first time.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

3D Gaussian Splatting (3DGS) has made significant strides in real-time 3D scene reconstruction, but faces memory scalability issues in high-resolution scenarios. To address this, we propose Hierarchical Gaussian Splatting (HRGS), a memory-efficient framework with hierarchical block-level optimization. First, we generate a global, coarse Gaussian representation from low-resolution data. Then, we partition the scene into multiple blocks, refining each block with high-resolution data. The partitioning involves two steps: Gaussian partitioning, where irregular scenes are normalized into a bounded cubic space with a uniform grid for task distribution, and training data partitioning, where only relevant observations are retained for each block. By guiding block refinement with the coarse Gaussian prior, we ensure seamless Gaussian fusion across adjacent blocks. To reduce computational demands, we introduce Importance-Driven Gaussian Pruning (IDGP), which computes importance scores for each Gaussian and removes those with minimal contribution, speeding up convergence and reducing memory usage. Additionally, we incorporate normal priors from a pretrained model to enhance surface reconstruction quality. Our method enables high-quality, high-resolution 3D scene reconstruction even under memory constraints. Extensive experiments on three benchmarks show that HRGS achieves state-of-the-art performance in high-resolution novel view synthesis (NVS) and surface reconstruction tasks.

Problem

• Memory-efficient high-resolution 3D reconstruction
• Hierarchical block-level optimization for Gaussian Splatting
• Importance-Driven Gaussian Pruning to reduce computational demands

Innovation

• Hierarchical block-level optimization for memory efficiency
• Importance-Driven Gaussian Pruning to reduce computation
• Normal priors enhance surface reconstruction quality