GeoPurify: A Data-Efficient Geometric Distillation Framework for Open-Vocabulary 3D Segmentation

2025-10-02

AI Summary

This work addresses projection noise, fragmented predictions, and geometric inconsistency arising when transferring 2D vision-language model (VLM) features to 3D semantic segmentation. We propose a geometry-aware lightweight knowledge distillation framework. Methodologically, we design a self-supervised 3D teacher model to implicitly encode geometric priors, and introduce a point-wise feature affinity network coupled with a geometry-guided pooling module to achieve label-free feature denoising and geometric consistency optimization. The lightweight student network requires only minimal 3D supervision—approximately 1.5% of annotated data. Evaluated on mainstream benchmarks including ScanNetv2, our approach achieves state-of-the-art or superior performance at significantly reduced annotation cost. To the best of our knowledge, this is the first method to simultaneously achieve high accuracy, strong geometric consistency, and high data efficiency in open-vocabulary 3D segmentation.

Technology Category

• Computer Vision: Segmentation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Recent attempts to transfer features from 2D Vision-Language Models (VLMs) to 3D semantic segmentation expose a persistent trade-off. Directly projecting 2D features into 3D yields noisy and fragmented predictions, whereas enforcing geometric coherence necessitates costly training pipelines and large-scale annotated 3D data. We argue that this limitation stems from the dominant segmentation-and-matching paradigm, which fails to reconcile 2D semantics with 3D geometric structure. The geometric cues are not eliminated during the 2D-to-3D transfer but remain latent within the noisy and view-aggregated features. To exploit this property, we propose GeoPurify that applies a small Student Affinity Network to purify 2D VLM-generated 3D point features using geometric priors distilled from a 3D self-supervised teacher model. During inference, we devise a Geometry-Guided Pooling module to further denoise the point cloud and ensure the semantic and structural consistency. Benefiting from latent geometric information and the learned affinity network, GeoPurify effectively mitigates the trade-off and achieves superior data efficiency. Extensive experiments on major 3D benchmarks demonstrate that GeoPurify achieves or surpasses state-of-the-art performance while utilizing only about 1.5% of the training data. Our codes and checkpoints are available at [https://github.com/tj12323/GeoPurify](https://github.com/tj12323/GeoPurify).

Problem

• Mitigating noisy 3D segmentation from 2D feature projection
• Enforcing geometric coherence without large annotated datasets
• Reconciling 2D semantics with 3D structure efficiently

Innovation

• Uses Student Affinity Network to purify 3D features
• Distills geometric priors from self-supervised teacher model
• Employs Geometry-Guided Pooling for semantic consistency