ProtoGuard-guided PROPEL: Class-Aware Prototype Enhancement and Progressive Labeling for Incremental 3D Point Cloud Segmentation

2025-04-02

AI Summary

To address three key challenges in class-incremental semantic segmentation of 3D point clouds—catastrophic forgetting, confusion among visually similar classes, and misclassification induced by long-tailed label distributions—we propose ProtoGuard, a prototype guarding mechanism, and PROPEL, a progressive pseudo-label optimization framework. ProtoGuard mitigates forgetting via class-aware geometric-semantic joint prototype modeling and attention-based feature aggregation. PROPEL refines pseudo-labels iteratively by jointly leveraging density-aware spatial distribution and semantic similarity, thereby enhancing discriminability between old and new classes. Evaluated under the five-step incremental setting on S3DIS, our method achieves 20.39% mIoU—significantly outperforming prior approaches—and is the first to systematically balance forgetting suppression, fine-grained class discrimination, and robust adaptation to long-tailed class distributions.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

3D point cloud semantic segmentation technology has been widely used. However, in real-world scenarios, the environment is evolving. Thus, offline-trained segmentation models may lead to catastrophic forgetting of previously seen classes. Class-incremental learning (CIL) is designed to address the problem of catastrophic forgetting. While point clouds are common, we observe high similarity and unclear boundaries between different classes. Meanwhile, they are known to be imbalanced in class distribution. These lead to issues including misclassification between similar classes and the long-tail problem, which have not been adequately addressed in previous CIL methods. We thus propose ProtoGuard and PROPEL (Progressive Refinement Of PsEudo-Labels). In the base-class training phase, ProtoGuard maintains geometric and semantic prototypes for each class, which are combined into prototype features using an attention mechanism. In the novel-class training phase, PROPEL inherits the base feature extractor and classifier, guiding pseudo-label propagation and updates based on density distribution and semantic similarity. Extensive experiments show that our approach achieves remarkable results on both the S3DIS and ScanNet datasets, improving the mIoU of 3D point cloud segmentation by a maximum of 20.39% under the 5-step CIL scenario on S3DIS.

Problem

• Addresses catastrophic forgetting in 3D point cloud segmentation
• Resolves misclassification due to high class similarity
• Mitigates class imbalance and long-tail distribution issues

Innovation

• Class-aware prototype enhancement using attention mechanism
• Progressive pseudo-labeling based on density distribution
• Combined geometric-semantic features for incremental learning