Sort-free Gaussian Splatting via Weighted Sum Rendering

2024-10-24

arXiv.org

AI Summary

3D Gaussian Splatting (3DGS) suffers from high rendering overhead, poor mobile GPU performance, and “pop”-like temporal artifacts due to its reliance on view-dependent, non-commutative alpha compositing and explicit point sorting. Method: We propose a sorting-free, differentiable weighted-sum rendering paradigm that approximates non-commutative alpha blending via a commutative weighted sum—eliminating viewpoint-dependent sorting entirely. We further introduce generalized Gaussian modeling and mobile-GPU-aware optimizations, including memory-access coalescing and compute-efficient kernel fusion. Contribution/Results: Our method achieves image quality on par with original 3DGS while significantly improving efficiency: it delivers an average 1.23× speedup on mobile GPUs and enables real-time rendering on resource-constrained devices. Crucially, it fully resolves sorting-induced visual popping artifacts, establishing the first practical, high-fidelity 3DGS deployment for mobile platforms.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Recently, 3D Gaussian Splatting (3DGS) has emerged as a significant advancement in 3D scene reconstruction, attracting considerable attention due to its ability to recover high-fidelity details while maintaining low complexity. Despite the promising results achieved by 3DGS, its rendering performance is constrained by its dependence on costly non-commutative alpha-blending operations. These operations mandate complex view dependent sorting operations that introduce computational overhead, especially on the resource-constrained platforms such as mobile phones. In this paper, we propose Weighted Sum Rendering, which approximates alpha blending with weighted sums, thereby removing the need for sorting. This simplifies implementation, delivers superior performance, and eliminates the"popping"artifacts caused by sorting. Experimental results show that optimizing a generalized Gaussian splatting formulation to the new differentiable rendering yields competitive image quality. The method was implemented and tested in a mobile device GPU, achieving on average $1.23 imes$ faster rendering.

Problem

• Eliminates costly sorting in 3D Gaussian Splatting rendering
• Reduces computational overhead on resource-constrained platforms
• Addresses popping artifacts from view-dependent sorting operations

Innovation

• Weighted Sum Rendering replaces alpha blending
• Eliminates sorting for faster performance
• Optimized for mobile GPU efficiency