Novel Class Discovery for Point Cloud Segmentation via Joint Learning of Causal Representation and Reasoning

2025-10-15

AI Summary

This work addresses zero-shot 3D point cloud novel class discovery (NCD) segmentation: segmenting unlabeled novel classes using supervision solely from annotated base classes. Methodologically, we propose a novel paradigm grounded in structural causal models (SCMs), which employs causal representation learning to disentangle confounding factors, constructs a cross-class graph structure to model causal dependencies between base and novel classes, and integrates deconfounded feature disentanglement with counterfactual reasoning. Our key contribution is the first systematic integration of causal inference into 3D open-world segmentation, enabling interpretable, cross-class knowledge transfer. Evaluated on multiple 3D and 2D NCD benchmarks, our approach achieves state-of-the-art performance in segmentation accuracy while providing superior causal interpretability compared to existing methods.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Computer Vision: 3D Computer Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

In this paper, we focus on Novel Class Discovery for Point Cloud Segmentation (3D-NCD), aiming to learn a model that can segment unlabeled (novel) 3D classes using only the supervision from labeled (base) 3D classes. The key to this task is to setup the exact correlations between the point representations and their base class labels, as well as the representation correlations between the points from base and novel classes. A coarse or statistical correlation learning may lead to the confusion in novel class inference. lf we impose a causal relationship as a strong correlated constraint upon the learning process, the essential point cloud representations that accurately correspond to the classes should be uncovered. To this end, we introduce a structural causal model (SCM) to re-formalize the 3D-NCD problem and propose a new method, i.e., Joint Learning of Causal Representation and Reasoning. Specifically, we first analyze hidden confounders in the base class representations and the causal relationships between the base and novel classes through SCM. We devise a causal representation prototype that eliminates confounders to capture the causal representations of base classes. A graph structure is then used to model the causal relationships between the base classes' causal representation prototypes and the novel class prototypes, enabling causal reasoning from base to novel classes. Extensive experiments and visualization results on 3D and 2D NCD semantic segmentation demonstrate the superiorities of our method.

Problem

• Segmenting novel 3D classes using only labeled base class supervision
• Establishing causal correlations between point representations and class labels
• Eliminating confounders to enable accurate novel class inference

Innovation

• Joint learning of causal representation and reasoning
• Structural causal model to eliminate hidden confounders
• Graph structure modeling causal relationships between classes