A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation

2025-08-13

AI Summary

This work systematically investigates the modeling capacity and technical pathways of 3D Gaussian Splatting (3DGS) for semantic-level downstream tasks—including segmentation, editing, and generation—addressing the gap between its geometric reconstruction strength and semantic understanding capability. Method: Leveraging 3DGS’s explicit, compact, and differentiable representation, we propose the first unified design principles and a taxonomy of learning paradigms tailored for semantic understanding, along with a cross-task analysis of supervision strategy evolution. Our approach integrates 2D foundation model priors and NeRF-inspired geometric constraints within an explicit scene representation framework, enabling efficient geometry–semantics joint modeling via differentiable rendering and self-supervised learning. Contribution/Results: We establish a unified evaluation benchmark on mainstream datasets, release open-source code and resource libraries, and demonstrate that 3DGS can be effectively extended beyond reconstruction toward semantic understanding and generative applications.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

3D Gaussian Splatting (3DGS) has recently emerged as a powerful alternative to Neural Radiance Fields (NeRF) for 3D scene representation, offering high-fidelity photorealistic rendering with real-time performance. Beyond novel view synthesis, the explicit and compact nature of 3DGS enables a wide range of downstream applications that require geometric and semantic understanding. This survey provides a comprehensive overview of recent progress in 3DGS applications. It first introduces 2D foundation models that support semantic understanding and control in 3DGS applications, followed by a review of NeRF-based methods that inform their 3DGS counterparts. We then categorize 3DGS applications into segmentation, editing, generation, and other functional tasks. For each, we summarize representative methods, supervision strategies, and learning paradigms, highlighting shared design principles and emerging trends. Commonly used datasets and evaluation protocols are also summarized, along with comparative analyses of recent methods across public benchmarks. To support ongoing research and development, a continually updated repository of papers, code, and resources is maintained at https://github.com/heshuting555/Awesome-3DGS-Applications.

Problem

• Surveying 3D Gaussian Splatting applications in segmentation, editing, and generation
• Comparing 3DGS with NeRF for 3D scene representation
• Reviewing datasets and evaluation protocols for 3DGS methods

Innovation

• 3D Gaussian Splatting for real-time rendering
• Integration of 2D foundation models
• Applications in segmentation, editing, generation