Continual Learning in 3D Point Clouds: Employing Spectral Techniques for Exemplar Selection

2024-09-12

arXiv.org

AI Summary

To mitigate catastrophic forgetting in continual learning for 3D point clouds, this paper proposes CL3D—the first framework to introduce spectral clustering into exemplar selection. It jointly selects class-representative prototypes across three spaces—raw coordinates, local (1024-D) geometric features, and global semantic features—using a geometry-aware, differentiable non-Euclidean distance metric. By integrating multi-scale geometric structural information, CL3D significantly enhances prototype discriminability and compactness. On ModelNet40 and ShapeNet, it achieves state-of-the-art accuracy while reducing memory overhead to 50% of baseline methods; on ScanNet, it improves accuracy by 4.1% with only 28% memory consumption. Key contributions are: (i) the first spectral-clustering-based prototype selection mechanism for 3D continual learning; (ii) a differentiable, geometry-aware distance modeling tailored to point cloud characteristics; and (iii) a lightweight, multi-space feature-coordinated rehearsal strategy.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

We introduce a novel framework for Continual Learning in 3D object classification. Our approach, CL3D, is based on the selection of prototypes from each class using spectral clustering. For non-Euclidean data such as point clouds, spectral clustering can be employed as long as one can define a distance measure between pairs of samples. Choosing the appropriate distance measure enables us to leverage 3D geometric characteristics to identify representative prototypes for each class. We explore the effectiveness of clustering in the input space (3D points), local feature space (1024-dimensional points), and global feature space. We conduct experiments on the ModelNet40, ShapeNet, and ScanNet datasets, achieving state-of-the-art accuracy exclusively through the use of input space features. By leveraging the combined input, local, and global features, we have improved the state-of-the-art on ModelNet and ShapeNet, utilizing nearly half the memory used by competing approaches. For the challenging ScanNet dataset, our method enhances accuracy by 4.1% while consuming just 28% of the memory used by our competitors, demonstrating the scalability of our approach.

Problem

• Develops a framework for continual learning in 3D object classification.
• Uses spectral clustering to select prototypes from non-Euclidean point cloud data.
• Improves accuracy and reduces memory usage on major 3D datasets.

Innovation

• Spectral clustering for prototype selection
• Leveraging 3D geometric characteristics
• Memory-efficient continual learning framework