BEA-GS: BEyond RAdiance Supervision in 3DGS for Precise Object Extraction

2026-05-10

AI Summary

Existing 3D Gaussian splatting methods struggle to achieve precise object-level editing and extraction due to the absence of explicit geometric optimization. This work introduces, for the first time, a geometry refinement strategy that operates without photometric supervision, jointly optimizing the positions and shapes of both visible and occluded Gaussians during training to accurately align with semantic boundaries. Built upon the 3D Gaussian splatting framework, our approach leverages a gradient propagation mechanism combining differentiable rasterization and non-rasterized pathways to formulate two geometric constraint loss functions. Experimental results across four datasets demonstrate that our method consistently outperforms twelve state-of-the-art approaches on six boundary segmentation metrics, achieving significantly improved object boundary accuracy.

Abstract

Most Gaussian Splatting techniques that provide a 3D semantic representation of the scene do not optimize the underlying 3D geometry, making object-level editing or asset extraction challenging. Recent methods, such as COBGS, Trace3D, ObjectGS, acknowledge this limitation and propose approaches that modify the scene's geometry to represent the underlying semantics. We advance this concept further by proposing a novel solution that provides near perfect boundaries in object extraction. We do so by introducing two new losses in the optimization that take care of: 1) a loss that modifies the geometry of visible Gaussians to respect semantic boundaries, and 2) a loss that adjusts the geometry of non-visible Gaussians that appear once the object is extracted. Our first loss propagates gradients directly through the rasterization, allowing for seamless integration within the optimization of the Gaussian parameters. The second loss also propagates gradients to Gaussian parameters but does so without passing through the rasterization, enabling modification of the scene's geometry even when little transmittance reaches a Gaussian (partial or non-visible). Exhaustive comparisons with 12 state of the art methods across 4 datasets, using six metrics, demonstrate that our approach produces overall the best boundary segmentation to date.

Problem

• 3D Gaussian Splatting
• object extraction
• semantic boundaries
• 3D geometry optimization
• scene representation

Innovation

• Gaussian Splatting
• object extraction
• semantic boundaries
• geometry optimization
• differentiable rendering