Generative Data Augmentation for Object Point Cloud Segmentation

2025-05-23

AI Summary

To address the scarcity of annotated data in point cloud segmentation—where conventional geometric augmentations lack diversity and state-of-the-art 3D generative models produce point clouds without point-level semantic labels—this paper introduces the first part-aware 3D diffusion generative model tailored for segmentation. We propose a three-stage generative augmentation framework: (1) conditional generation of high-fidelity point clouds with part-level semantic annotations; (2) diffusion-process-guided pseudo-label confidence estimation and filtering; and (3) reweighted training using generated samples. Evaluated on both synthetic and real-world medical point cloud datasets, our method significantly outperforms traditional augmentation and leading semi-supervised and self-supervised approaches. Under few-shot settings, it achieves a 5.2% improvement in mean Intersection-over-Union (mIoU), demonstrating substantial gains in label-efficient segmentation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Data augmentation is widely used to train deep learning models to address data scarcity. However, traditional data augmentation (TDA) typically relies on simple geometric transformation, such as random rotation and rescaling, resulting in minimal data diversity enrichment and limited model performance improvement. State-of-the-art generative models for 3D shape generation rely on the denoising diffusion probabilistic models and manage to generate realistic novel point clouds for 3D content creation and manipulation. Nevertheless, the generated 3D shapes lack associated point-wise semantic labels, restricting their usage in enlarging the training data for point cloud segmentation tasks. To bridge the gap between data augmentation techniques and the advanced diffusion models, we extend the state-of-the-art 3D diffusion model, Lion, to a part-aware generative model that can generate high-quality point clouds conditioned on given segmentation masks. Leveraging the novel generative model, we introduce a 3-step generative data augmentation (GDA) pipeline for point cloud segmentation training. Our GDA approach requires only a small amount of labeled samples but enriches the training data with generated variants and pseudo-labeled samples, which are validated by a novel diffusion-based pseudo-label filtering method. Extensive experiments on two large-scale synthetic datasets and a real-world medical dataset demonstrate that our GDA method outperforms TDA approach and related semi-supervised and self-supervised methods.

Problem

• Enhance data diversity for point cloud segmentation
• Generate labeled 3D shapes using diffusion models
• Improve model performance with limited labeled data

Innovation

• Extends Lion diffusion model for part-aware generation
• Introduces 3-step generative data augmentation pipeline
• Uses diffusion-based pseudo-label filtering method