A Novel Decomposed Feature-Oriented Framework for Open-Set Semantic Segmentation on LiDAR Data

2025-03-14

AI Summary

This paper addresses the challenge of simultaneously achieving accurate closed-set semantic segmentation and reliable unknown-class identification in LiDAR point cloud open-set semantic segmentation. To this end, we propose a decoupled dual-decoder network: one decoder is dedicated to closed-set semantic segmentation, while the other independently models feature representations of unknown classes. We further introduce an open-set discrimination mechanism based on feature-space anomaly detection, integrated with a multi-objective joint loss function and bird’s-eye-view (BEV) voxelization encoding. Evaluated on SemanticKITTI and nuScenes, our method significantly outperforms state-of-the-art approaches—improving unknown-class detection rate by 12.6% while maintaining superior mean intersection-over-union (mIoU) for known classes. Notably, it is the first work to achieve architectural and optimization-level synergy between open-set discrimination and closed-set segmentation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Semantic segmentation is a key technique that enables mobile robots to understand and navigate surrounding environments autonomously. However, most existing works focus on segmenting known objects, overlooking the identification of unknown classes, which is common in real-world applications. In this paper, we propose a feature-oriented framework for open-set semantic segmentation on LiDAR data, capable of identifying unknown objects while retaining the ability to classify known ones. We design a decomposed dual-decoder network to simultaneously perform closed-set semantic segmentation and generate distinctive features for unknown objects. The network is trained with multi-objective loss functions to capture the characteristics of known and unknown objects. Using the extracted features, we introduce an anomaly detection mechanism to identify unknown objects. By integrating the results of close-set semantic segmentation and anomaly detection, we achieve effective feature-driven LiDAR open-set semantic segmentation. Evaluations on both SemanticKITTI and nuScenes datasets demonstrate that our proposed framework significantly outperforms state-of-the-art methods. The source code will be made publicly available at https://github.com/nubot-nudt/DOSS.

Problem

• Identifies unknown objects in LiDAR data segmentation.
• Proposes a dual-decoder network for known and unknown object classification.
• Introduces anomaly detection for effective open-set semantic segmentation.

Innovation

• Decomposed dual-decoder network for LiDAR segmentation
• Multi-objective loss for known and unknown objects
• Anomaly detection mechanism for unknown object identification