DV-Matcher: Deformation-based Non-Rigid Point Cloud Matching Guided by Pre-trained Visual Features

2024-08-16

AI Summary

This paper addresses the challenging problem of dense correspondence estimation between non-rigid point clouds—particularly under realistic conditions including near-isometric or heterogeneous shapes, partial overlap, and noise. We propose an end-to-end deep matching framework that requires neither meshing nor manual annotations. Our method uniquely integrates global semantic priors from pretrained vision models into geometric feature learning for point clouds and introduces a deformation-guided module that jointly optimizes extrinsic alignment accuracy and feature discriminability. By enabling cross-modal fusion of visual and geometric features and modeling differentiable deformation constraints, our approach achieves robust and generalizable dense matching. Extensive experiments on standard benchmarks demonstrate state-of-the-art performance, with significant improvements in robustness to non-rigid deformation, occlusion, and noise compared to existing methods.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Learning with Manifolds

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

In this paper, we present DV-Matcher, a novel learning-based framework for estimating dense correspondences between non-rigidly deformable point clouds. Learning directly from unstructured point clouds without meshing or manual labelling, our framework delivers high-quality dense correspondences, which is of significant practical utility in point cloud processing. Our key contributions are two-fold: First, we propose a scheme to inject prior knowledge from pre-trained vision models into geometric feature learning, which effectively complements the local nature of geometric features with global and semantic information; Second, we propose a novel deformation-based module to promote the extrinsic alignment induced by the learned correspondences, which effectively enhances the feature learning. Experimental results show that our method achieves state-of-the-art results in matching non-rigid point clouds in both near-isometric and heterogeneous shape collection as well as more realistic partial and noisy data.

Problem

• Estimates dense correspondences between non-rigidly deformable point clouds.
• Injects pre-trained vision model knowledge into geometric feature learning.
• Enhances feature learning with a novel deformation-based alignment module.

Innovation

• Pre-trained visual features enhance geometric learning
• Deformation-based module improves extrinsic alignment
• Achieves state-of-the-art in non-rigid point cloud matching