Intrinsic 4D Gaussian Segmentation from Scene Cues

2026-06-16

AI Summary

This work addresses the high cost and mask-quality dependency of existing dynamic 4D Gaussian scene segmentation methods that rely on multi-view 2D foundation model masks. We propose Intrinsic-GS, the first intrinsic segmentation approach that requires neither training nor external masks. Instead, it leverages only geometric and dynamic cues inherent to Gaussian primitives—including appearance, orientation, scale, deformation trajectories, and non-learned rendering boundaries—to construct a sparse affinity graph, followed by automatic clustering via Leiden community detection. Our method achieves mIoU scores of 0.746 and 0.575 on Neu3D and HyperNeRF, respectively. Notably, using geometric cues alone, it attains 0.902 mIoU on Neu3D—comparable to SAM-supervised methods—and operates 12.5× faster than current pipelines on HyperNeRF, significantly improving both efficiency and robustness.

Abstract

Dynamic 4D Gaussian Splatting reconstructs deforming scenes with high fidelity and is increasingly adopted as a representation for dynamic 3D scenes. Putting such a scene to use, for editing, manipulation or motion analysis, first requires segmenting it: grouping the Gaussian primitives into coherent objects. Current pipelines obtain this grouping by importing 2D masks from foundation models such as SAM and lifting or distilling them into the Gaussian representation. In dynamic scenes these masks must be generated across many frames and views, which is costly, and the resulting segmentation can depend strongly on the quality and consistency of those external masks. We ask how much object-level structure can instead be recovered from the Gaussians themselves, and propose Intrinsic-GS, a training-free, mask-free method that builds a sparse affinity graph over Gaussian primitives from appearance, orientation, scale, deformation-trajectory and non-learned rendered-boundary cues. The graph is partitioned with Leiden community detection, requiring no foundation model and no learned feature field. On the standard 4D Gaussian segmentation benchmarks, Neu3D and HyperNeRF, Intrinsic-GS recovers substantial object structure without mask supervision, reaching 0.746 mIoU on Neu3D and 0.575 on HyperNeRF; on Neu3D, a geometry-only variant reaches 0.902 mIoU, matching SAM-supervised TRASE. On HyperNeRF, Intrinsic-GS runs 12.5x faster than the mask-generation and feature-rendering stages used by mask-supervised pipelines. These results suggest that much of the segmentation signal is already encoded in the Gaussians themselves, offering a fast, mask-free direction for 3D and 4D Gaussian segmentation that may also point toward more generalizable, robust segmentation in settings where external masks are unreliable or expensive.

Problem

• 4D Gaussian Segmentation
• Dynamic Scene
• Object Segmentation
• Mask-free
• Gaussian Splatting

Innovation

• Intrinsic Segmentation
• 4D Gaussian Splatting
• Mask-Free
• Affinity Graph
• Leiden Community Detection