Exploring Scene Affinity for Semi-Supervised LiDAR Semantic Segmentation

2024-08-21

AI Summary

This work addresses the underutilization of unlabeled data in semi-supervised LiDAR semantic segmentation by proposing Affinity-aware Instance Scene (AIScene) modeling, which jointly captures intra-scene consistency and inter-scene correlations. Within a teacher-student framework, we introduce a point-level erasing strategy to enforce forward/backward propagation consistency and pioneer patch-level mixup augmentation at both scene- and instance-granularities to enhance the semantic diversity and robustness of pseudo-labels. Extensive experiments on SemanticKITTI and nuScenes under four semi-supervised settings demonstrate consistent superiority over state-of-the-art methods. Notably, under the extreme 1% labeled-data setting, our approach achieves absolute mIoU improvements of +1.9% on SemanticKITTI and +2.1% on nuScenes, validating the efficacy of AIScene modeling for driving-scene understanding under severe label scarcity.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Computer Vision: Scene Analysis & Understanding
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

This paper explores scene affinity (AIScene), namely intra-scene consistency and inter-scene correlation, for semi-supervised LiDAR semantic segmentation in driving scenes. Adopting teacher-student training, AIScene employs a teacher network to generate pseudo-labeled scenes from unlabeled data, which then supervise the student network's learning. Unlike most methods that include all points in pseudo-labeled scenes for forward propagation but only pseudo-labeled points for backpropagation, AIScene removes points without pseudo-labels, ensuring consistency in both forward and backward propagation within the scene. This simple point erasure strategy effectively prevents unsupervised, semantically ambiguous points (excluded in backpropagation) from affecting the learning of pseudo-labeled points. Moreover, AIScene incorporates patch-based data augmentation, mixing multiple scenes at both scene and instance levels. Compared to existing augmentation techniques that typically perform scene-level mixing between two scenes, our method enhances the semantic diversity of labeled (or pseudo-labeled) scenes, thereby improving the semi-supervised performance of segmentation models. Experiments show that AIScene outperforms previous methods on two popular benchmarks across four settings, achieving notable improvements of 1.9% and 2.1% in the most challenging 1% labeled data.

Problem

• Explores scene affinity for semi-supervised LiDAR semantic segmentation
• Uses teacher-student training with pseudo-labeled scenes for consistency
• Enhances semantic diversity via patch-based multi-scene data augmentation

Innovation

• Teacher-student training with pseudo-labeled scenes
• Point erasure for consistent propagation
• Patch-based multi-scene data augmentation