Robust Prior-Guided Segmentation for Editable 3D Gaussian Splatting

2026-05-15

AI Summary

This work addresses the limited robustness of 3D Gaussian Splatting in semantic segmentation, which hinders object-level editing due to view inconsistency and coarse masks in existing approaches. To overcome these limitations, the authors propose a prior-guided label reassignment mechanism that leverages high-fidelity 2D masks generated by SAM-HQ and enforces multi-view consistency constraints to assign semantic labels to 3D Gaussians. The method significantly enhances segmentation boundary fidelity and preserves fine structural details, achieving state-of-the-art 3D object segmentation accuracy. It enables high-fidelity, real-time object manipulation—including removal, extraction, and recoloring—and demonstrates strong practical utility in applications such as virtual reality and robotics.

Abstract

3D Gaussian Splatting (3D-GS) enables real-time 3D scene reconstruction but lacks robust segmentation for editing tasks such as object removal, extraction, and recoloring. Existing approaches that lift 2D segmentations to the 3D domain suffer from view inconsistencies and coarse masks. In this paper, we propose a novel framework that leverages the Segment Anything Model High Quality (SAM-HQ) to generate accurate 2D masks, addressing the limitations of the standard SAM in boundary fidelity and fine-structure preservation. To achieve robust 3D segmentation of any target object in a given scene, we introduce a prior-guided label reassignment method that assigns labels to 3D Gaussians by enforcing multiview consistency with learned priors. Our approach achieves state-of-the-art segmentation accuracy and enables interactive, real-time object editing while maintaining high visual fidelity. Qualitative results demonstrate superior boundary preservation and practical utility in Virtual Reality (VR) and robotics, advancing 3D scene editing.

Problem

• 3D Gaussian Splatting
• robust segmentation
• view inconsistency
• coarse masks
• 3D scene editing

Innovation

• 3D Gaussian Splatting
• SAM-HQ
• prior-guided segmentation
• multiview consistency
• real-time editing