Clustering is back: Reaching state-of-the-art LiDAR instance segmentation without training

2025-03-17

AI Summary

Existing LiDAR point cloud panoptic segmentation methods rely heavily on large-scale instance annotations and supervised training, leading to high annotation costs and poor generalization. Method: This paper proposes a zero-training, zero-instance-annotation geometric-semantic joint clustering framework that requires only semantic labels. By integrating spatial connectivity analysis with semantic consistency constraints, it achieves interpretable, hyperparameter-free, plug-and-play real-time instance segmentation. Results: Our method matches supervised state-of-the-art (SOTA) performance on SemanticKITTI and nuScenes; as a plug-and-play instance head on SemanticKITTI, it outperforms all published unsupervised and weakly supervised approaches. It runs at real-time speed on single-threaded CPU inference. To our knowledge, this is the first work achieving SOTA panoptic segmentation under fully unsupervised, zero-training, and zero-instance-annotation settings—establishing a new paradigm for efficient, robust, and low-cost deployment in large-scale LiDAR point cloud scenes.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Unsupervised & Self-Supervised Learning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Panoptic segmentation of LiDAR point clouds is fundamental to outdoor scene understanding, with autonomous driving being a primary application. While state-of-the-art approaches typically rely on end-to-end deep learning architectures and extensive manual annotations of instances, the significant cost and time investment required for labeling large-scale point cloud datasets remains a major bottleneck in this field. In this work, we demonstrate that competitive panoptic segmentation can be achieved using only semantic labels, with instances predicted without any training or annotations. Our method achieves performance comparable to current state-of-the-art supervised methods on standard benchmarks including SemanticKITTI and nuScenes, and outperforms every publicly available method on SemanticKITTI as a drop-in instance head replacement, while running in real-time on a single-threaded CPU and requiring no instance labels. Our method is fully explainable, and requires no learning or parameter tuning. Code is available at https://github.com/valeoai/Alpine/

Problem

• Achieves LiDAR panoptic segmentation without training or annotations
• Reduces cost and time by eliminating manual instance labeling
• Provides real-time performance on single-threaded CPU without instance labels

Innovation

• Uses semantic labels for panoptic segmentation
• Predicts instances without training or annotations
• Runs real-time on single-threaded CPU