SAD-GS: Learning Reliable 3D Semantic Gaussian Fields via Dynamic Geo-Semantic Anchoring

2026-06-28

AI Summary

Existing open-vocabulary 3D semantic Gaussian field methods relying on multi-view 2D supervision are prone to view-dependent artifacts, leading to semantic drift, mask boundary leakage, and identity switching, which cause multi-view inconsistency and error accumulation. To address these issues, this work proposes a dynamic geometry–semantic anchoring mechanism that distills view-invariant semantic identities via Semantic Anchor Distillation (SAD) and introduces a Geometry–Semantic Feedback Loop (GSFL) coupled with a three-gate conservative update rule. This enables 3D-guided self-correction of 2D supervision and robust spatial mask assignment. Evaluated on LERF-OVS, 3D-OVS, and Mip-NeRF360 datasets, the method significantly outperforms existing approaches, achieving state-of-the-art overall performance in open-vocabulary localization and semantic segmentation tasks, thereby demonstrating its effectiveness and robustness.

Abstract

Open-vocabulary 3D semantic Gaussian field learning relies on multi-view 2D supervision, whose semantic targets and spatial assignments are often unreliable. Across varying viewpoints, view-dependent features cause semantic identity drift, while propagated tracker masks introduce boundary leakage and identity switches. Directly optimizing against these unreliable 2D targets forces the 3D representation to absorb multi-view contradictions, leading to severe error accumulation. To resolve this limitation, we propose SAD-GS, a framework for learning reliable 3D semantic Gaussian fields via dynamic geo-semantic anchoring. Specifically, Semantic Anchor Distillation (SAD) distills per-view visual embeddings into consensus text anchors to establish a viewpoint-invariant semantic identity. Concurrently, the Geo-Semantic Feedback Loop (GSFL) leverages the evolving 3D field to actively filter tracker anomalies and refine spatial mask assignments via a conservative three-gate update rule. Extensive evaluations on LERF-OVS, 3D-OVS, and Mip-NeRF360 show that SAD-GS consistently achieves the best overall performance in both open-vocabulary localization and semantic segmentation. These comprehensive improvements validate the effectiveness and robustness of dynamic geo-semantic anchoring for reliable 3D semantic Gaussian field learning.

Problem

• open-vocabulary 3D semantic
• multi-view 2D supervision
• semantic identity drift
• boundary leakage
• error accumulation

Innovation

• Semantic Anchor Distillation
• Geo-Semantic Feedback Loop
• 3D Semantic Gaussian Fields
• Open-vocabulary Learning
• Dynamic Geo-Semantic Anchoring