COB-GS: Clear Object Boundaries in 3DGS Segmentation Based on Boundary-Adaptive Gaussian Splitting

2025-03-25

AI Summary

In 3D Gaussian Splatting (3DGS), Gaussian primitives exhibit inherent volumetric extent and lack semantic constraints, causing them to bleed across object boundaries and resulting in ambiguous segmentation edges. Method: We propose a semantics-guided boundary-adaptive Gaussian splitting method that jointly leverages semantic gradient statistics and texture-aware inpainting to dynamically identify object boundaries and split Gaussians precisely along them—avoiding visual degradation from conventional pruning. Additionally, we introduce a joint semantic-visual loss function that simultaneously optimizes segmentation accuracy and rendering fidelity. Results: Our approach achieves over 8% improvement in mean Intersection-over-Union (mIoU) across multiple benchmarks. Crucially, it preserves sharp segmentation boundaries even under inaccurate pre-trained masks, significantly enhancing segmentation robustness and 3D scene reconstruction fidelity.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Sampling/Simulation-based Search
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Accurate object segmentation is crucial for high-quality scene understanding in the 3D vision domain. However, 3D segmentation based on 3D Gaussian Splatting (3DGS) struggles with accurately delineating object boundaries, as Gaussian primitives often span across object edges due to their inherent volume and the lack of semantic guidance during training. In order to tackle these challenges, we introduce Clear Object Boundaries for 3DGS Segmentation (COB-GS), which aims to improve segmentation accuracy by clearly delineating blurry boundaries of interwoven Gaussian primitives within the scene. Unlike existing approaches that remove ambiguous Gaussians and sacrifice visual quality, COB-GS, as a 3DGS refinement method, jointly optimizes semantic and visual information, allowing the two different levels to cooperate with each other effectively. Specifically, for the semantic guidance, we introduce a boundary-adaptive Gaussian splitting technique that leverages semantic gradient statistics to identify and split ambiguous Gaussians, aligning them closely with object boundaries. For the visual optimization, we rectify the degraded suboptimal texture of the 3DGS scene, particularly along the refined boundary structures. Experimental results show that COB-GS substantially improves segmentation accuracy and robustness against inaccurate masks from pre-trained model, yielding clear boundaries while preserving high visual quality. Code is available at https://github.com/ZestfulJX/COB-GS.

Problem

• Improving 3DGS segmentation accuracy at object boundaries
• Resolving blurry boundaries in interwoven Gaussian primitives
• Balancing semantic and visual optimization in 3DGS refinement

Innovation

• Boundary-adaptive Gaussian splitting technique
• Joint optimization of semantic and visual information
• Rectification of degraded suboptimal texture