S2AM3D: Scale-controllable Part Segmentation of 3D Point Cloud

2025-11-30

AI Summary

Addressing the challenges of poor generalization and inconsistent multi-view predictions in 3D point cloud part-level segmentation, this paper proposes S2AM3D—a novel framework integrating point-wise part encoding with scale-aware prompt decoding to jointly leverage 2D segmentation priors and 3D geometric consistency supervision. Methodologically, it introduces multi-view 2D feature extraction, native 3D contrastive learning, point-level consistency aggregation, and a differentiable scale prompting mechanism to ensure cross-view segmentation consistency and controllable granularity. To support robust training, we construct a high-quality dataset comprising over 100,000 samples. Extensive experiments demonstrate that S2AM3D achieves state-of-the-art performance across multiple benchmarks, significantly enhancing robustness for complex structures and parts exhibiting large-scale variations.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Large-scale security measurements

Abstract

Part-level point cloud segmentation has recently attracted significant attention in 3D computer vision. Nevertheless, existing research is constrained by two major challenges: native 3D models lack generalization due to data scarcity, while introducing 2D pre-trained knowledge often leads to inconsistent segmentation results across different views. To address these challenges, we propose S2AM3D, which incorporates 2D segmentation priors with 3D consistent supervision. We design a point-consistent part encoder that aggregates multi-view 2D features through native 3D contrastive learning, producing globally consistent point features. A scale-aware prompt decoder is then proposed to enable real-time adjustment of segmentation granularity via continuous scale signals. Simultaneously, we introduce a large-scale, high-quality part-level point cloud dataset with more than 100k samples, providing ample supervision signals for model training. Extensive experiments demonstrate that S2AM3D achieves leading performance across multiple evaluation settings, exhibiting exceptional robustness and controllability when handling complex structures and parts with significant size variations.

Problem

• Addresses generalization issues in 3D part segmentation due to data scarcity
• Resolves inconsistent segmentation from multi-view 2D priors in 3D models
• Enables real-time adjustment of segmentation granularity via scale signals

Innovation

• Aggregates multi-view 2D features via 3D contrastive learning
• Adjusts segmentation granularity with continuous scale signals
• Introduces large-scale dataset for part-level point cloud training