3DMPE: 3D Multi-Perspective Embedding

2026-07-06

AI Summary

This work addresses the challenges of missing correspondences and incomplete geometric information in point cloud reconstruction from partially observed multi-view inputs. The authors propose a training-free optimization method that jointly recovers the 3D point cloud and its cross-view projection mappings. Built upon an extended multi-view synchronized embedding framework, the approach integrates variable projection, geometric constraints, and visibility modeling, making it applicable to both fixed and variable projection settings without requiring category-specific priors. Experiments on ShapeNet and Pix3D demonstrate that the method robustly reconstructs partial multi-view point clouds, consistently outperforming existing non-learning baselines across Chamfer distance, Earth Mover’s Distance (EMD), and Reconstruction Overlap Accuracy (ROA) metrics.

Abstract

We study 3D point cloud reconstruction from multiple partially observed 2D projections. Given two or more projections of an unknown 3D point cloud, together with cross-view point correspondences and visibility information, our goal is to recover a consistent 3D configuration when different views contain different subsets of points. We propose 3D Multi-Perspective Embedding (3DMPE), an optimization-based, training-free method that reconstructs the 3D point cloud and, in the variable-projection setting, jointly estimates the projection maps. 3DMPE extends Multi-Perspective Simultaneous Embedding to accommodate missing points and incomplete pairwise distance information across views. We consider both fixed-projection and variable-projection settings. Unlike learning-based reconstruction methods that infer shape from raw images and often depend on training data, 3DMPE operates on geometric observations with established correspondences and does not require category-specific training. Experiments on ShapeNet and Pix3D evaluate reconstruction quality using Chamfer Distance, Earth Mover Distance, and RMSE-Optimize-Align (ROA), and examine the effects of initialization, the number of views, point visibility, and several noise regimes, including noisy distances and erroneous correspondences. The results demonstrate that 3DMPE can effectively reconstruct point clouds from partial multi-view geometric observations.

Problem

• 3D point cloud reconstruction
• multi-view projection
• partial observation
• cross-view correspondence
• missing points

Innovation

• 3D reconstruction
• multi-view embedding
• point cloud
• correspondence-based
• optimization-based