MEGS$^{2}$: Memory-Efficient Gaussian Splatting via Spherical Gaussians and Unified Pruning

2025-09-07

AI Summary

3D Gaussian Splatting (3DGS) suffers from high rendering memory overhead, hindering deployment on resource-constrained edge devices. Existing compression methods primarily target storage reduction, neglecting the critical GPU memory bottleneck during rendering. To address this, we propose a unified rendering-memory compression framework that jointly optimizes both the number of primitives and the parameter count per primitive. Specifically, we replace spherical harmonics for color modeling with lightweight, arbitrarily oriented spherical Gaussian lobes, and introduce a unified soft-pruning mechanism that formulates primitive and lobe pruning as an end-to-end differentiable constrained optimization problem, enabling dynamic structural simplification during training. Experiments demonstrate that our method achieves a 50% reduction in static GPU memory and a 40% reduction in peak rendering memory—without compromising rendering quality—thereby significantly enhancing the practicality of 3DGS on memory-limited edge platforms.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

3D Gaussian Splatting (3DGS) has emerged as a dominant novel-view synthesis technique, but its high memory consumption severely limits its applicability on edge devices. A growing number of 3DGS compression methods have been proposed to make 3DGS more efficient, yet most only focus on storage compression and fail to address the critical bottleneck of rendering memory. To address this problem, we introduce MEGS$^{2}$, a novel memory-efficient framework that tackles this challenge by jointly optimizing two key factors: the total primitive number and the parameters per primitive, achieving unprecedented memory compression. Specifically, we replace the memory-intensive spherical harmonics with lightweight arbitrarily-oriented spherical Gaussian lobes as our color representations. More importantly, we propose a unified soft pruning framework that models primitive-number and lobe-number pruning as a single constrained optimization problem. Experiments show that MEGS$^{2}$ achieves a 50% static VRAM reduction and a 40% rendering VRAM reduction compared to existing methods, while maintaining comparable rendering quality.

Problem

• Reducing high memory consumption in 3D Gaussian Splatting for edge devices
• Addressing rendering memory bottleneck beyond storage compression methods
• Optimizing primitive count and parameters per primitive simultaneously

Innovation

• Spherical Gaussians for color representation
• Unified pruning framework for optimization
• Joint optimization of primitive parameters