PointCubeNet: 3D Part-level Reasoning with 3x3x3 Point Cloud Blocks

2025-11-10

AI Summary

This work addresses 3D part-level understanding of point clouds without part-level annotations. We propose PointCubeNet, a multimodal framework that partitions point clouds into 3×3×3 voxel blocks and leverages local text-point cloud alignment to generate pseudo-part labels, enabling end-to-end unsupervised training via a designed local contrastive loss. The architecture features dual global-local branches: the global branch captures holistic object semantics, while the local branch performs fine-grained, block-level part reasoning. To our knowledge, this is the first method achieving interpretable and structurally consistent 3D part segmentation and semantic labeling under purely unsupervised settings. Experiments on benchmarks including ShapeNet demonstrate substantial improvements over existing unsupervised approaches. The generated part-level predictions exhibit high geometric plausibility and semantic coherence, significantly enhancing comprehensive 3D object understanding.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this paper, we propose PointCubeNet, a novel multi-modal 3D understanding framework that achieves part-level reasoning without requiring any part annotations. PointCubeNet comprises global and local branches. The proposed local branch, structured into 3x3x3 local blocks, enables part-level analysis of point cloud sub-regions with the corresponding local text labels. Leveraging the proposed pseudo-labeling method and local loss function, PointCubeNet is effectively trained in an unsupervised manner. The experimental results demonstrate that understanding 3D object parts enhances the understanding of the overall 3D object. In addition, this is the first attempt to perform unsupervised 3D part-level reasoning and achieves reliable and meaningful results.

Problem

• Achieving part-level 3D reasoning without part annotations
• Enabling unsupervised analysis of point cloud sub-regions
• Enhancing overall 3D object understanding through part comprehension

Innovation

• Uses 3x3x3 local blocks for part-level analysis
• Employs unsupervised pseudo-labeling method for training
• Combines global and local branches for 3D understanding