Gaussians on a Diet: High-Quality Memory-Bounded 3D Gaussian Splatting Training

2026-04-21

AI Summary

This work addresses the prohibitively high peak memory consumption of 3D Gaussian Splatting (3DGS) during training, which stems from uncontrolled densification and hinders deployment on edge devices. To overcome this limitation, the authors propose a memory-constrained training framework that maintains near-constant low memory usage by dynamically balancing Gaussian growth and pruning throughout optimization. The key innovation lies in an adaptive Gaussian compensation mechanism that effectively harmonizes densification and pruning, surpassing the capabilities of existing post-processing approaches. Experimental results across diverse real-world scenes demonstrate that the method significantly outperforms current techniques, enabling direct training on a Jetson AGX Xavier with an 80% reduction in peak memory while preserving rendering quality comparable to the original 3DGS.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

3D Gaussian Splatting (3DGS) has revolutionized novel view synthesis with high-quality rendering through continuous aggregations of millions of 3D Gaussian primitives. However, it suffers from a substantial memory footprint, particularly during training due to uncontrolled densification, posing a critical bottleneck for deployment on memory-constrained edge devices. While existing methods prune redundant Gaussians post-training, they fail to address the peak memory spikes caused by the abrupt growth of Gaussians early in the training process. To solve the training memory consumption problem, we propose a systematic memory-bounded training framework that dynamically optimizes Gaussians through iterative growth and pruning. In other words, the proposed framework alternates between incremental pruning of low-impact Gaussians and strategic growing of new primitives with an adaptive Gaussian compensation, maintaining a near-constant low memory usage while progressively refining rendering fidelity. We comprehensively evaluate the proposed training framework on various real-world datasets under strict memory constraints, showing significant improvements over existing state-of-the-art methods. Particularly, our proposed method practically enables memory-efficient 3DGS training on NVIDIA Jetson AGX Xavier, achieving similar visual quality with up to 80% lower peak training memory consumption than the original 3DGS.

Problem

• 3D Gaussian Splatting
• memory consumption
• training memory
• memory-constrained devices
• densification

Innovation

• memory-bounded training
• 3D Gaussian Splatting
• dynamic pruning and growth
• adaptive Gaussian compensation
• edge deployment