GenPC: Zero-shot Point Cloud Completion via 3D Generative Priors

2025-02-27

AI Summary

Existing point cloud completion methods rely on synthetic training data and exhibit poor generalization to out-of-distribution real-world scans. This paper proposes GenPC, a zero-shot framework that reconstructs unseen real-world point clouds without task-specific training. Our approach leverages internet-scale, single-view image–driven 3D generative priors—previously unexplored for point cloud completion—via two novel components: (1) a depth prompting module and a geometry-preserving fusion module, enabling effective transfer of pretrained visual-3D knowledge; and (2) a joint optimization strategy integrating depth-image guidance, differentiable pose-and-scale refinement, and implicit shape alignment to ensure structural fidelity and pose-adaptive reconstruction. GenPC significantly outperforms supervised methods across multiple benchmarks, achieving state-of-the-art generalization on real-world scans while maintaining superior geometric consistency.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Existing point cloud completion methods, which typically depend on predefined synthetic training datasets, encounter significant challenges when applied to out-of-distribution, real-world scans. To overcome this limitation, we introduce a zero-shot completion framework, termed GenPC, designed to reconstruct high-quality real-world scans by leveraging explicit 3D generative priors. Our key insight is that recent feed-forward 3D generative models, trained on extensive internet-scale data, have demonstrated the ability to perform 3D generation from single-view images in a zero-shot setting. To harness this for completion, we first develop a Depth Prompting module that links partial point clouds with image-to-3D generative models by leveraging depth images as a stepping stone. To retain the original partial structure in the final results, we design the Geometric Preserving Fusion module that aligns the generated shape with input by adaptively adjusting its pose and scale. Extensive experiments on widely used benchmarks validate the superiority and generalizability of our approach, bringing us a step closer to robust real-world scan completion.

Problem

• Zero-shot point cloud completion
• Real-world scan reconstruction
• 3D generative priors utilization

Innovation

• Zero-shot point cloud completion
• Depth Prompting module integration
• Geometric Preserving Fusion module