Relation-Centric Open-Vocabulary 3D Gaussian Segmentation

2026-07-01

AI Summary

This work addresses the challenges of language grounding and boundary precision in open-vocabulary 3D Gaussian segmentation by introducing, for the first time, a segmentation framework centered on modeling pairwise Gaussian relations. It constructs a sparse relation graph through multi-view mask fusion and view contribution weighting, computes affinities using low-dimensional descriptors, and leverages a hierarchical clustering tree to support multi-granularity semantic queries. The method operates without per-scene optimization, substantially improving both efficiency and segmentation quality. It achieves state-of-the-art performance on open-vocabulary benchmarks, with its accelerated variant running 50 times faster than existing instance-feature optimization–based approaches.

Abstract

Open-vocabulary 3D Gaussian segmentation is challenging because it requires language understanding for diverse queries and accurate separation of Gaussians along object boundaries. Prior approaches either embed language knowledge into individual Gaussians to improve query responsiveness or optimize per-Gaussian instance features to encode object identity. However, these strategies may produce noisy Gaussian segmentations or rely on cost-inefficient per-scene optimization. We propose PairGS, a framework that reframes Gaussian segmentation as modeling pairwise relations between Gaussians. 3D Gaussian representations provide rich signals for relation estimation, such as view contribution weights and multi-view mask evidence. By leveraging these cues, PairGS explicitly constructs a relation graph for segmentation without a heavy optimization process. PairGS first proposes sparse edge candidates using low-dimensional descriptors, computes precise pairwise affinities only on those candidates, and builds a hierarchical cluster tree for multi-granular querying. It achieves state-of-the-art results on open-vocabulary 3D Gaussian segmentation benchmarks, while the fast variant is 50x faster than optimization-based instance-feature approaches.

Problem

• open-vocabulary
• 3D Gaussian segmentation
• language understanding
• object boundary separation
• Gaussian segmentation noise

Innovation

• Pairwise Relation Modeling
• 3D Gaussian Segmentation
• Open-Vocabulary
• Relation Graph
• Hierarchical Clustering