Distill, Diffuse, and Semanticize (DDS): Annotation-Free 3D Scene Understanding Based on Multi-Granularity Distillation and Graph-Diffusion-Based Segmentation

2026-05-08

AI Summary

This work addresses key challenges in unsupervised 3D scene semantic understanding—namely, inconsistent regional semantics, inefficient global clustering, and the absence of interpretable semantic categories—by proposing a novel unsupervised framework that integrates multi-granularity knowledge distillation, superpoint graph diffusion, and a segmentation-clustering association mechanism. Without requiring any human-annotated 3D labels, the method enables efficient global semantic propagation and enforces object-level consistency. Notably, it is the first to assign interpretable semantic categories to 3D regions under a fully unsupervised setting. Evaluated on real-world datasets, the approach significantly outperforms existing methods, achieving absolute gains of 5.9% in overall accuracy (oAcc), 8.1% in mean class accuracy (mAcc), and 2.4% in mean Intersection over Union (mIoU).

Abstract

3D semantic scene understanding has broad applications in digital twins, autonomous driving, smart agriculture, and embodied perception. However, dense point-wise annotation for point clouds is extremely expensive, making fully supervised 3D semantic learning difficult to scale. Recent annotation-free methods can discover semantic regions without manual 3D labels, but they often suffer from weak object-level consistency, inefficient global grouping, and category-agnostic segmented regions. We propose an annotation-free 3D scene semantic understanding method based on multi-granularity distillation and graph-diffusion-based segmentation. The proposed method first leverages structured visual knowledge guidance and superpoint graph diffusion to perform efficient global semantic propagation, alleviating the problem of inconsistent region-level semantics. It then conducts semantic inference through segmentation-cluster association, assigning interpretable category names to segmented 3D regions and improving the overall effectiveness of annotation-free 3D semantic understanding. Extensive experiments on real-world datasets demonstrate the effectiveness of the proposed framework. Compared with the advanced existing annotation-free baselines, our method improves oAcc, mAcc, and mIoU by 5.9%, 8.1%, and 2.4% at most, respectively. These results highlight the promise of the proposed framework for scalable annotation-free 3D scene understanding, especially in real-world scenarios requiring both object segmentation and semantic recognition.

Problem

• 3D semantic scene understanding
• annotation-free
• point cloud segmentation
• semantic consistency
• category-agnostic segmentation

Innovation

• annotation-free
• graph diffusion
• multi-granularity distillation
• 3D semantic segmentation
• semantic propagation