LLM-Guided Taxonomy and Hierarchical Uncertainty for 3D Point CLoud Active Learning

2025-05-25

AI Summary

Active learning for 3D point cloud semantic segmentation under extremely low annotation budgets remains challenging due to the difficulty of balancing spatial diversity and hierarchical label consistency. Method: This paper proposes a semantic hierarchy-aware active learning framework. It leverages large language models (LLMs) to automatically construct a hierarchical semantic taxonomy and introduces a cross-level uncertainty projection and recursive propagation mechanism to integrate hierarchical priors into sample selection. The method synergistically combines LLM-based prompt engineering, hierarchical uncertainty modeling, and a dedicated 3D point cloud active learning architecture. Contribution/Results: Evaluated on S3DIS and ScanNet v2, the framework achieves a +4.0% mIoU improvement using only 0.02% labeled data—substantially outperforming state-of-the-art methods. It establishes a scalable, structured active learning paradigm for resource-constrained 3D point cloud segmentation.

Technology Category

• Machine Learning: Active Learning
• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: 3D Computer Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

We present a novel active learning framework for 3D point cloud semantic segmentation that, for the first time, integrates large language models (LLMs) to construct hierarchical label structures and guide uncertainty-based sample selection. Unlike prior methods that treat labels as flat and independent, our approach leverages LLM prompting to automatically generate multi-level semantic taxonomies and introduces a recursive uncertainty projection mechanism that propagates uncertainty across hierarchy levels. This enables spatially diverse, label-aware point selection that respects the inherent semantic structure of 3D scenes. Experiments on S3DIS and ScanNet v2 show that our method achieves up to 4% mIoU improvement under extremely low annotation budgets (e.g., 0.02%), substantially outperforming existing baselines. Our results highlight the untapped potential of LLMs as knowledge priors in 3D vision and establish hierarchical uncertainty modeling as a powerful paradigm for efficient point cloud annotation.

Problem

• Integrates LLMs for 3D point cloud active learning
• Generates hierarchical semantic taxonomies using LLMs
• Improves annotation efficiency under low budgets

Innovation

• LLMs generate hierarchical semantic taxonomies
• Recursive uncertainty projection across hierarchy levels
• Label-aware point selection for 3D scenes