A Survey on 3D Gaussian Splatting

2024-01-08

arXiv.org

AI Summary

This work presents a systematic survey of 3D Gaussian Splatting (3D GS), clarifying its theoretical foundations as an explicit radiance field representation paradigm, its key technical advancements, and its practical applicability boundaries. We introduce the first 3D GS knowledge graph, elucidating its intrinsic mechanisms—explicit spatial parameterization, differentiable rasterization, and strong editability—and rigorously distinguishing it from implicit NeRF-based approaches. Methodologically, we integrate adaptive Gaussian optimization, density-aware regularization, and multi-view geometric constraints to enable end-to-end training and real-time rendering (>100 FPS). Comprehensive experiments benchmark state-of-the-art models across reconstruction accuracy, inference speed, and editing flexibility, revealing critical limitations—including low geometric fidelity and weak support for dynamic scenes. Finally, we identify promising future directions: scalability to large-scale scenes, physical plausibility enforcement, and cross-modal integration with vision-language or sensor-fusion frameworks.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

3D Gaussian splatting (GS) has emerged as a transformative technique in explicit radiance field and computer graphics. This innovative approach, characterized by the use of millions of learnable 3D Gaussians, represents a significant departure from mainstream neural radiance field approaches, which predominantly use implicit, coordinate-based models to map spatial coordinates to pixel values. 3D GS, with its explicit scene representation and differentiable rendering algorithm, not only promises real-time rendering capability but also introduces unprecedented levels of editability. This positions 3D GS as a potential game-changer for the next generation of 3D reconstruction and representation. In the present paper, we provide the first systematic overview of the recent developments and critical contributions in the domain of 3D GS. We begin with a detailed exploration of the underlying principles and the driving forces behind the emergence of 3D GS, laying the groundwork for understanding its significance. A focal point of our discussion is the practical applicability of 3D GS. By enabling unprecedented rendering speed, 3D GS opens up a plethora of applications, ranging from virtual reality to interactive media and beyond. This is complemented by a comparative analysis of leading 3D GS models, evaluated across various benchmark tasks to highlight their performance and practical utility. The survey concludes by identifying current challenges and suggesting potential avenues for future research. Through this survey, we aim to provide a valuable resource for both newcomers and seasoned researchers, fostering further exploration and advancement in explicit radiance field.

Problem

• 3D Gaussian splatting in radiance fields
• Real-time rendering and scene editability
• Comparative analysis of 3D GS models

Innovation

• Millions of learnable 3D Gaussians
• Explicit scene representation
• Differentiable rendering algorithm