CoE: Deep Coupled Embedding for Non-Rigid Point Cloud Correspondences

2024-12-07

AI Summary

To address the challenge of jointly modeling semantic and geometric information in non-rigid point cloud registration—complicated by clutter, occlusion, noise, and large deformations—this paper proposes the Coupled Embedding (CoE) paradigm. CoE employs an end-to-end neural network to learn per-point high-dimensional embeddings, jointly optimizing a contrastive loss that unifies semantic similarity and local geometric structure. Crucially, CoE achieves high-accuracy dense correspondences solely via nearest-neighbor search in the embedding space, eliminating the need for explicit deformation modeling. Evaluated on multiple non-rigid matching benchmarks, CoE establishes new state-of-the-art performance, demonstrating significantly improved robustness to noise, truncation, and large deformations. Moreover, the learned embeddings exhibit strong generalization capability in downstream tasks such as segmentation, validating their discriminative and transferable representation power.

Technology Category

• Search and Optimization: Learning to Search
• Computer Vision: Multi-modal Vision
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The interest in matching non-rigidly deformed shapes represented as raw point clouds is rising due to the proliferation of low-cost 3D sensors. Yet, the task is challenging since point clouds are irregular and there is a lack of intrinsic shape information. We propose to tackle these challenges by learning a new shape representation -- a per-point high dimensional embedding, in an embedding space where semantically similar points share similar embeddings. The learned embedding has multiple beneficial properties: it is aware of the underlying shape geometry and is robust to shape deformations and various shape artefacts, such as noise and partiality. Consequently, this embedding can be directly employed to retrieve high-quality dense correspondences through a simple nearest neighbor search in the embedding space. Extensive experiments demonstrate new state-of-the-art results and robustness in numerous challenging non-rigid shape matching benchmarks and show its great potential in other shape analysis tasks, such as segmentation.

Problem

• 3D Deformable Point Clouds
• Matching Problem
• Noisy Data

Innovation

• Deformable 3D Point Cloud Matching
• High-dimensional Shape Feature Mapping
• Robustness to Noise and Incompleteness