LabelGS: Label-Aware 3D Gaussian Splatting for 3D Scene Segmentation

2025-08-27

AI Summary

While 3D Gaussian Splatting (3DGS) enables efficient rendering and high-fidelity scene reconstruction, it lacks native support for semantic segmentation, limiting its applicability in scene understanding tasks. To address this, we propose a label-aware 3DGS framework. Our method introduces: (1) cross-view consistent semantic masks and occlusion-aware primary Gaussian labeling to resolve label ambiguity and conflict; (2) a label propagation mechanism leveraging Gaussian projection filtering and 2D semantic priors; and (3) a stochastic region sampling strategy to enhance training stability and segmentation consistency. Evaluated at 1440×1080 resolution, our approach trains 22× faster than Feature-3DGS while achieving significantly higher semantic segmentation accuracy than state-of-the-art methods. To the best of our knowledge, this is the first end-to-end 3DGS-based framework that simultaneously delivers both computational efficiency and high segmentation precision.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Semi-Supervised Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Security and Privacy: Data transparency and provenance

Abstract

3D Gaussian Splatting (3DGS) has emerged as a novel explicit representation for 3D scenes, offering both high-fidelity reconstruction and efficient rendering. However, 3DGS lacks 3D segmentation ability, which limits its applicability in tasks that require scene understanding. The identification and isolating of specific object components is crucial. To address this limitation, we propose Label-aware 3D Gaussian Splatting (LabelGS), a method that augments the Gaussian representation with object label.LabelGS introduces cross-view consistent semantic masks for 3D Gaussians and employs a novel Occlusion Analysis Model to avoid overfitting occlusion during optimization, Main Gaussian Labeling model to lift 2D semantic prior to 3D Gaussian and Gaussian Projection Filter to avoid Gaussian label conflict. Our approach achieves effective decoupling of Gaussian representations and refines the 3DGS optimization process through a random region sampling strategy, significantly improving efficiency. Extensive experiments demonstrate that LabelGS outperforms previous state-of-the-art methods, including Feature-3DGS, in the 3D scene segmentation task. Notably, LabelGS achieves a remarkable 22X speedup in training compared to Feature-3DGS, at a resolution of 1440X1080. Our code will be at https://github.com/garrisonz/LabelGS.

Problem

• Enabling 3D segmentation capability in Gaussian Splatting representation
• Lifting 2D semantic labels to 3D Gaussian representations
• Resolving occlusion and label conflicts in 3D scene segmentation

Innovation

• Cross-view consistent semantic masks
• Occlusion Analysis Model optimization
• Random region sampling strategy