Point Cloud Based Scene Segmentation: A Survey

2025-03-16

AI Summary

3D object detection yields coarse-grained bounding boxes, insufficient for fine-grained autonomous driving decisions (e.g., navigation, lane-changing). To address this, this work systematically surveys state-of-the-art point cloud semantic segmentation methods for autonomous driving and, for the first time, unifies them into three paradigms: projection-based, 3D intrinsic, and hybrid approaches. Emphasizing dense and fine-grained environmental understanding, we analyze key techniques—including spherical/Bird’s Eye View (BEV) CNNs, voxel- and point-based sparse convolutions (e.g., PointPillars, KPConv), multimodal fusion, and knowledge distillation—and highlight synthetic data’s critical role in mitigating real-world annotation scarcity. We quantitatively benchmark mainstream methods on SemanticKITTI and nuScenes, reporting mean Intersection-over-Union (mIoU) and inference efficiency. The empirical evaluation provides actionable guidance for algorithm selection and practical deployment in autonomous systems.

Technology Category

• Computer Vision: Segmentation
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Autonomous driving is a safety-critical application, and it is therefore a top priority that the accompanying assistance systems are able to provide precise information about the surrounding environment of the vehicle. Tasks such as 3D Object Detection deliver an insufficiently detailed understanding of the surrounding scene because they only predict a bounding box for foreground objects. In contrast, 3D Semantic Segmentation provides richer and denser information about the environment by assigning a label to each individual point, which is of paramount importance for autonomous driving tasks, such as navigation or lane changes. To inspire future research, in this review paper, we provide a comprehensive overview of the current state-of-the-art methods in the field of Point Cloud Semantic Segmentation for autonomous driving. We categorize the approaches into projection-based, 3D-based and hybrid methods. Moreover, we discuss the most important and commonly used datasets for this task and also emphasize the importance of synthetic data to support research when real-world data is limited. We further present the results of the different methods and compare them with respect to their segmentation accuracy and efficiency.

Problem

• Enhancing autonomous driving safety through precise environment understanding.
• Improving 3D scene segmentation for detailed environmental information.
• Reviewing state-of-the-art methods in point cloud semantic segmentation.

Innovation

• Point Cloud Semantic Segmentation for autonomous driving
• Categorization into projection-based, 3D-based, hybrid methods
• Use of synthetic data to supplement limited real-world data