Generalized Zero-Shot Learning for Point Cloud Segmentation with Evidence-Based Dynamic Calibration

2025-09-10

AI Summary

Generalized zero-shot 3D point cloud segmentation suffers from severe prediction bias: models exhibit overconfidence toward seen classes, substantially degrading unseen-class recognition performance—exacerbated by limited training point clouds. To address this, we propose E3DPC-GZSL, the first method to introduce evidential deep learning into point cloud segmentation. It quantifies per-point predictive uncertainty via evidence-based uncertainty estimation and employs a dynamically calibrated stacking factor to adaptively reweight seen/unseen class probabilities. Additionally, a learnable text enhancement module is incorporated to improve cross-modal semantic alignment between point clouds and textual class descriptions. The framework is end-to-end trainable without requiring auxiliary annotations. Evaluated on ScanNet v2 and S3DIS, E3DPC-GZSL achieves state-of-the-art performance, with significant gains in unseen-class mIoU and effective mitigation of classification bias.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: 3D Computer Vision
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Generalized zero-shot semantic segmentation of 3D point clouds aims to classify each point into both seen and unseen classes. A significant challenge with these models is their tendency to make biased predictions, often favoring the classes encountered during training. This problem is more pronounced in 3D applications, where the scale of the training data is typically smaller than in image-based tasks. To address this problem, we propose a novel method called E3DPC-GZSL, which reduces overconfident predictions towards seen classes without relying on separate classifiers for seen and unseen data. E3DPC-GZSL tackles the overconfidence problem by integrating an evidence-based uncertainty estimator into a classifier. This estimator is then used to adjust prediction probabilities using a dynamic calibrated stacking factor that accounts for pointwise prediction uncertainty. In addition, E3DPC-GZSL introduces a novel training strategy that improves uncertainty estimation by refining the semantic space. This is achieved by merging learnable parameters with text-derived features, thereby improving model optimization for unseen data. Extensive experiments demonstrate that the proposed approach achieves state-of-the-art performance on generalized zero-shot semantic segmentation datasets, including ScanNet v2 and S3DIS.

Problem

• Reducing biased predictions in 3D point cloud segmentation
• Addressing overconfidence towards seen classes in zero-shot learning
• Improving uncertainty estimation for unseen class classification

Innovation

• Evidence-based uncertainty estimator integration
• Dynamic calibrated stacking factor adjustment
• Learnable parameters merged with text features